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Foreword 


The scientists who discovered semiconductors in laboratory studies in the first half of 
the twentieth century, which gave rise to microelectronics in the second half ofthe twen- 
tieth century, could hardly have imagined that by using semiconductors in computers 
they would trigger a worldwide technology revolution, now called digitalisation. Now, 
some 80 years after the discovery of semiconductors, atthe end ofa pandemic in which 
the state had to restrict personal freedoms in almost every country to prevent the health 
care system from collapsing, the use of digitalisation has not only accelerated, but has 
also helped to some extent to limit the economic disadvantages of such restrictions. This 
book, “The World We Want to Live In - Compendium of Digitalisation, Digital Net- 
works, and Artificial Intelligence” (original title “Wie wir leben wollen”) is comprehen- 
sive and spans from the first part on “Humans and Digital Technology” to the second 
part on the “Necessity of Legal Design” (with a sub-chapter on autonomous weapons), 
and finally to the third part on the “Political Design of Digitalisation” All chapters are 
written by scientists with a high level of experience, and the internal discussion has also 
led to initial recommendations in the fourth and last part on the “Responsibility of Sci- 
ence”. One sub-aspect would have deserved more attention: technical innovation, which 
is also progressing faster and faster through digitalisation, is widening the gap between 
unrestricted use, which is often detrimental to at least parts of society, and the limitation 
of misuse through standards as well as laws. Social debate, and therefore legislation, is 
increasingly lagging behind technical progress. How can this gap be narrowed? Another 
central problem is the global inequality that is growing even further with digitalisation, 
because fewer and fewer countries are contributing to progress in digitalisation and 
so-called artificial intelligence. Some industrialised countries, but especially emerging 
economies and developing countries in general, are left behind. The problem mentioned 
is addressed in the book, but not yet in sufficient detail considering its importance. My 
recommendation: Please read this book and - stimulated by it - intensify the debate 
with scientific colleagues, interested citizens, and decision-makers on all subtopics! 


Prof. Dr. Dr. h.c. mult. Hartmut Graßl, 
Chairman of the Federation of German Scientists e.V. (VDW) 


Counting People? People Are Counting! 
A Preface 


What is actually the opposite of digitalisation? Analogisation? 
The opposite of networking? Isolation? 
The opposite of artificial intelligence? Human stupidity? 


At least about the latter, we have the evidence-supported statement of a respected scien- 
tist called Albert Einstein: “Two things are infinite, the universe and human stupidity, 
but I am not quite sure about the universe yet?’ 

But is human stupidity really the opposite of artificial intelligence (AI)? And if so, 
would artificial intelligence have to be infinitely small to pass as the opposite of infinitely 
large human stupidity? Are networking and isolation opposites? Or does not networking 
by means of modern communication technologies almost automatically lead to isolation 
- or, conversely, have some Tekkis who feel isolated invented networking technologies 
in order to no longer be so alone? And is there really a hard difference between digital 
and analogue? Are they just two ends of a continuum? 

Personally, I am not as sure about the answers. But in science, the questions often 
enough put us on the right track - even if it still takes some hard work to arrive at the 
answers that are considered valid for a particular time or culture. In this respect, I am 
very pleased that the volume presented here, “The World We Want to Live In - Compen- 
dium of Digitalisation, Digital Networks, and Artificial Intelligence” (original title “Wie 
wir leben wollen - Kompendium zu Technikfolgen von Digitalisierung, Vernetzung und 
Künstlicher Intelligenz”), raises many questions and that the authors do not shy away 
from one or the other answer! The annual conference of the Federation of German 
Scientists, which took place in October 2019 at the University of Applied Sciences in 
Berlin, already made the need for these topics and the discussion worthiness of many 
questions more than clear. So clear, in fact, that now almost two years later - and despite 
the constraints of the Covid 19 pandemic - an impressive compendium has emerged. 
I would like to sincerely thank all those involved and wish the articles in this book a 
broad, ready, and attentive readership! 

Digitalisation in its modern manifestations has become one of the defining issues 
of our time and of global importance, not just because of the Corona pandemic. It only 
goes through various cycles of attention - analogous to most of our other “humanity 
issues’, such as the finite nature of natural resources, man-made climate change, glo- 
balisation, health care, or even the global mobility of people, things, and ideas. In fact, 
digitalisation is centuries older than the extraordinarily stupid dictum of a German 


1 Incidentally, the authorship of this quotation is not entirely certain: although it is usually attributed to 
Albert Einstein, it is not really documented. 
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chancellor about the alleged “new territory of the Internet” from 2013 and also the 
justified mockery of it suggest. And even if it sometimes seems to us as if digitalisation 
with all its technologies, devices, global networks, etc. is rolling over us like a mighty 
tsunami of unknown origin, the historical truth is quite different: All elements of digi- 
talisation, including networking and artificial intelligence, were invented by people, are 
driven by them, or even slowed down once in a while. And their beginnings go back to 
the dawn of humankind. The term itself provides the first clue: “Digitus” is Latin and 
was the finger in ancient Rome. Because many people over the centuries have counted 
and calculated with their fingers, the English dictionary now provides the translations 
for “digit”: digit, number, finger, digit and toe. 

The central basic principle of digitalisation, the abstraction and description of phys- 
ical phenomena with the help of signs, is similarly old. While the first signs and scripts 
were still characterised by the need to depict the described objects pictorially, as can still 
be seen today in Egyptian hieroglyphics or Chinese character writing, for example, at 
some point mankind invented writing systems based on a relatively small set of charac- 
ters. Our current alphabet is an example of this. With only 26 letters and various special 
characters such as full stops, commas, etc., we can form an infinite number of concepts, 
sentences, and descriptions of anything and even put non-physical phenomena such 
as love, thoughts, or antimatter into words. The core of this perhaps most important 
human invention is to give things a name. 

This makes it possible to speak or write about things and phenomena without 
them having to be present themselves. The power of this is immediately obvious when 
one imagines, for example, how good it was to warn children or tribal members of 
sabre-toothed tigers without them having to stand in front of them and baring their 
teeth. But crafts and technology have also benefited from this possibility for thousands 
of years, which ultimately forms the core of all media technology, from cave paintings 
to books and television to augmented and virtual reality applications. Conceptually, 
the detachment from the physical presence of objects in conjunction with their media 
“doubling” is the core of all “tele-technology” and modern networking technologies. 

Sign and writing systems became much more economical when people began to ex- 
press the names and descriptions of things with abstracted signs that are completely de- 
tached from the appearance of things. This means that far fewer characters are needed, 
e.g. fewer than a hundred characters in our current writing system. Seen in this light, 
digitalisation is a time-honoured phenomenon originating tens of thousands of years 
ago when humans learned to count and spell. The fact that we could count, calculate, 
read, and write could well be placed alongside humour among the peculiarities of hu- 
mans that distinguish us from other living beings. 

The special thing about abstracted sign systems like our alphabet is that the number 
of signs is finite (preferably even quite small), that the signs are clearly distinguishable, 
do not occur simultaneously, and must not contradict each other. All sign systems that 
fulfil these conditions are equally powerful and can be translated into each other qua- 
si-automatically with the help of simple rules. Our everyday 100-character system is 
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therefore equivalent, for example, to a system of two characters, such as 0 and 1. This 
is why it has been possible since the 1940s to develop machines that are specifically 
designed to help us count and calculate. Most of these so-called computers (= Latin and 
English: “counters”) can now count and calculate better than most people, and with 
their help we can capture, process, store, or distribute writing much better than with all 
earlier technologies. Since the 1990s, the WorldWideWeb has added a steadily growing 
international infrastructure that makes it possible to send and exchange numbers, cal- 
culations, and all kinds of content around the globe in fractions of a second. 

The origins of these mediumistic concepts and techniques are thus almost as old 
as humanity and are perhaps even an essential feature of our becoming human. In any 
case, they are intimately connected with the history of humanity. This in turn means 
that we have also been familiar with virtually all the challenges of digitalisation and 
media use for millennials: The danger of misunderstanding when authors and recipi- 
ents of information are distant from each other; the possibilities of deception; misuse 
for defaming other people or societies; use for power interests and violence; spread 
of pornography, voyeurism, and sexual assault; dangers of addiction to the content or 
even some forms of media technology; exploitation for economic enrichment at the 
expense of others etc. 

The good part is that despite countless excesses, most societies and humanity as a 
whole have succeeded time and again over the millennia in using digital concepts and 
techniques to contain the dangers and make their benefits widely available. 

That is why it is essential, on the one hand, to constantly face up to the challenges 
and possible negative consequences of media and digitalisation technologies - as the 
articles in this volume do - and, on the other hand, to regularly look at what solutions 
previous societies or individuals have already developed. 

For me personally, in addition to looking at history, it often helps to also consider 
fiction. Therefore, to conclude this little preface, I will take the liberty of referring to two 
authors to whom I, as a graduate computer scientist from the last century, always like to 
return, Isaac Asimov and Stanislav Lem. Both have dealt with questions of digitalisation 
(which was not yet called that at the time), artificial intelligence, and networking in their 
own way, fortunately often with some humour and certainly sometimes with more depth 
than many current contributions. 

As an example, I would like to highlight Asimov’s robot cycle, whose individual 
stories partly come across as criminal science fiction but at their core almost always 
revolve around the question of how humans as inventors relate to robots as the in- 
vented. This is encapsulated and seemingly resolved in three so-called “robot laws”, 
which superficially establish the primacy of humans, but in their interpretation and 
delimitation anticipate many questions, for example, about autonomous driving or the 
limits of artificial intelligence. 

If you don't feel like rereading such old stories, you can see a surprisingly intelligent 
summary in Hollywood's “I Robot” from 2004 with Bridget Moynahan and Will Smith. 
Key question: If a robot / artificial intelligence concludes that humans tend to harm 
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themselves, wouldn't it be legitimate - and compatible with robot laws - to barrack 
humans to protect themselves from themselves? 

From Lem, I would like to highlight “The Washing Machine Tragedy”, a grotesque 
about washing machines that are at some point capable of many other activities in 
addition to their washing duties, can look like humans and are finally “bred up” so 
intelligently and emotionally that they sue for their “civil” rights before the Supreme 
Court. Again, at its core, is the question of the relationship between humans and their 
intelligent machines, spiced up with the economic interests of two large corporations, 
and sharpened to the limit of “Should I laugh or cry about this? Or should I rather think 
about it?” 

Ultimately, digitalisation is man-made, and the dangers or abuses result very much 
from technology, but even more so from human needs. And we know all about those! 

To return to the beginning, part of the message of the old sense of the word digitus 
to modern “digitalisation” could be not to abstract things so much that they become 
incomprehensible, or vice versa: the more we digitise, the more we have to make sure 
that things remain manageable and humanly controllable. 

This is because digitalisation only works when people count. In more ways than one. 

Or to take up the title of this volume: If we don’t care about “the world we want to 
live in’, others will - possibly machines... 


Prof. Dr. Carsten Busch, 
President ofthe University of Applied Sciences, Berlin 
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The World We Want to Live In — An Introduction 


Frank Schmiedchen 


We need to understand profoundly, where we stand and what the next steps will be of 
digitalisation, networks (or interconnectedness), and machine learning (artificial intelli- 
gence; AI) to make informed and wise decisions. Our most important questions maybe: 
What are the effects of the latest digital technologies in various applications and what 
are the consequences for me, for my country, and for humankind? 

Complex digital networks and artificial intelligence are fundamental innovations 
that permeate the economy and society in many fields, thus becoming the engine of a 
comprehensive, disruptive structural change. These innovations, being new technical 
approaches, are also the technological basis for a vast amount of present and future 
follow-up innovations that are changing our lives profoundly and at high speed. 

Despite numerous publications, the profound and multifaceted effects of advanced 
digital development are usually considered only in isolated areas, i.e. for specific so- 
cioeconomic, sociocultural, or sociotechnical areas. In a long-term study on media 
discourse in Germany, Fischer and Puschmann find that there is a one-sided focus on 
economic and technology-euphoric aspects of current digital developments. In contrast, 
other social issues are still discussed too rarely (cf. Puschmann/ Fischer, 2020; pp. 29ff). 
There is still a lack of diversity in terms of perspectives and actors represented (ibid.). 
The discourse is thus dominated by the assumption of a generally valid and broadly ac- 
cepted approval of technology-optimistic concepts for the future, which usually remain 
vague and are not critically questioned. 

Given the exponential increase in knowledge, the lack of scientific (e.g. neurolog- 
ical, [social] psychological, legal, or economic) analysis means that fewer and fewer 
people or institutions have a somewhat rudimentary understanding of recent digital 
developments. As a result, these few persons or institutions can increasingly determine 
further developments in an uncontrolled manner, in the sense of “the winner takes it all”. 
However, the greater challenge by far is the fact that a majority of publications to date 
explicitly or implicitly suggest that the continued pursuit of the chosen technological 
path is an inevitable fate for humanity. This culminates in the claim that the chosen 
technological path determines humankind’s evolutionary future, i.e. we are already in 
a so-called lock-in today. 

The advanced digital development poses societal challenges, but also existential 
risks for our human future. There are not many like the US documentary filmmaker 
and author James Barrat, who dig deeper on possible further developments. He con- 
cludes that a broad, social discussion about the fundamental connections and dangers 
of further digital development, especially of AI, is essential for humanity’s survival. 
Barrat therefore calls for comprehensive technology assessments as a necessity that he 
describes dramatically as “nor does this alter the fact that we will have just one chance to 
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establish a positive coexistence with beings whose intelligence is greater than our own” 
(Barrat, 2013, p. 267). Barrat has had an unusual impact with his 2013 book ‘The Last 
Invention, which was one of the triggers for Elon Musk’s decision to found Neuralink in 
2016, which is expected to build human-machine interface prototypes this year (2021) 
after a successful trial with a macaque monkey (Musk, 2021; Kelly, 2021). According 
to Elon Musk, one goal of founding Neuralink is to arm humanity through a symbiotic 
relationship with digital technology to counter future successful attacks by hostile and 
powerful AI (Hamilton, 2019). 

But even if this is classified as science fiction, since reaching the innovation threshold 
of the technological singularity is highly controversial in terms of the likelihood of its 
occurrence (from “in a few years” to “never”), there are numerous challenges that the 
advanced digital developments pose for our daily life. Especially since the outbreak of 
the Covid-19 pandemic most voices suggest that there is no alternative to the recent 
technology path. 

An example of the danger that has been growing for the past few years is the con- 
sideration of “The Great Reset”: a digital “Singaporeisation” of western democracies, 
i.e. the destruction of open societies through rampant prohibition policies that pretend 
to be necessary to implement the UN Sustainable Development Goals or the Paris Cli- 
mate Agreement (SDG 2030, Schwab/Mallert, 2020, WEF, 2020). Good intentions are 
suggested here (e.g. closing the digital divide, climate protection, diversity). However, 
the policies proposed are a nightmare smoothie of “1984” and “Brave New World” that 
would lead to digitally controlled autocratic regimes. These attempts are flanked by 
illiberal identity policies (cf. Fourest, 2020, Kastner/Susemichel, 2020, Wagenknecht, 
2021), that are supposed to define “right action, right speech and right thinking” even 
in democratic, open societies. Compliance with these is to be digitally monitored and 
sanctioned in a variety of ways (e.g., higher health insurance contributions for people 
without smart fitness bracelets). However, such a system would mean that “transparent” 
people would potentially be constantly monitored in a digital panopticon to see whether 
they are behaving correctly and appropriately (cf. Chapter 1). 

On the other hand, digital technologies have become indispensable tools of everyday 
life for most people around the world and are extremely useful in many ways: they 
simplify and speed up many tasks, connect people, reduce dangers, create convenience, 
are fun, and save lives. This makes them an enrichment of human life, especially when 
they are used for the common good. Fundamental criticism is counterproductive in 
view of the broad enthusiasm of most people worldwide for their digital access and the 
vehement demands of those who do not have (sufficient) access and want it. It is us, 
as citizens, workers, learners, consumers, or patients, who demand barrier-free access 
to more and more digitalisation and make extensive use of the digital media available. 
At least in the three technologically leading regions of the world, North America, East 
Asia, and Europe, people are largely convinced that the human future will be a digital 
one, whatever that may mean. The Covid-19 pandemic has once again dramatically 
accelerated this development. 
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Here is an excursus by Oliver Ponsold who discusses personal interaction with dig- 
ital media: 


A Personal Approach to the Digital World 
Oliver Ponsold 


For years, a new reality of life has been taking shape in our personal interaction 
with the digital: Through the convergence of advanced technical performance 
and available energy storage in pocket format, a digital personal environment is 
created where high-performance connectivity is reliably available, which perma- 
nently supports and influences our perceptions and evaluations of the environ- 
ment, and based on this, enables or channels short-term decisions and actions. 

As an example, service companies that arrange and deliver meals from 
third-party providers to customers offer a link to the actual service provider. 
Permanent use leads to service providers becoming more customer-oriented 
and learning from them, especially when extra data from the digital customer 
ego is also included. This reduces their entrepreneurial risk and tends to result 
in higher customer satisfaction, and thus loyalty. A dedicated, modern digitised 
manufacturing and distribution environment supports speed and consideration 
of customer-specific special requests, as well as iterative purchasing models and 
long-term customer loyalty in service delivery. Service providers earn extremely 
well from platforms that are ostensibly free of charge for the customer through 
enormous scaling effects and intelligent evaluation of collected usage data, the 
actual means of payment of the Internet. 

Finally, the natural person converges with their own virtual images on the 
Internet and the digital assistants they carry with them during long-term inten- 
sive use. The digital footprints left behind are evaluated by AI algorithms and 
used for targeted selection preferences and (product) recommendations, so 
that an enhanced positive and binding user experience is made possible. This 
includes, for example, the combination of e-commerce platforms with social 
networks, the circle of friends, and acquaintances. The resulting psychological 
and practical path dependencies and lock-in mechanisms are presented and 
discussed in detail in Chapter 3 of this book. 

Four blocks of questions can shed light on this in a very personal way: 


1. Is looking at the smartphone a constant daily ritual? How often and on what 
occasions do I use it? Do I often use different media at the same time and 
are they smartly connected? 

2. From which source or medium do I obtain information and do I dou- 
ble-check it? How quickly do I expect answers? What personal imprint 
profile do I leave behind in my research? 
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3. How often and what kind of life moments do I share on social networks 
and which and how many people do I follow on social networks, blogs, or 
podcasts? 

4. What would be the consequences if my virtual personality with access 
to email and all social networks would be stolen and used for criminal 
purposes? 


Such an analysis shows how deeply humans and digital technology are already 
closely interwoven. Digital technology enables us to access information in real 
time, which we validate and evaluate through personal use in the mirror of our 
knowledge and preferences. 


Since 2016, the Federation of German Scientists (VDW) has been intensively address- 
ing the consequences of networked digitalisation and artificial intelligence and has es- 
tablished a working group for this purpose. The group has published a “Policy Paper 
on the Asilomar Principles on Artificial Intelligence” in 2018 and in 2019 organised a 
conference on “The Ambivalences of the Digital, together with various German uni- 
versities, the German Trade Union Federation (DGB), and civil society, at the Berlin 
University of Applied Sciences. This compendium is another milestone in the VDW’s 
deliberations on digitalisation topics and forms the basis for further work. 

In view of the almost religious euphoria regarding the advancing use of digitalisa- 
tion, networking, and Al in practically all areas of life, it is a necessary and natural task 
of the VDW to point out underestimated or ignored scientifically and socially relevant 
and existential problems of this development, to provide approaches for technology 
assessments, and to make well-founded proposals for an ethically justifiable approach. 


e Who determines what is “good” when technology becomes increasingly perva- 
sive and affects all people, directly or at least indirectly, not just those who have 
consciously chosen to use it? 

e For example, do we have a consensus in the EU, USA, or ASEAN on what risks 
we are willing to accept in order to make our lives more and more comfortable? 

e How can such a consensus be achieved at the global level? 


We see the enormous potential for liberation that digitalisation already means for in- 
dividuals and societies if these possibilities of individual freedom and socio-ecological 
development are used. 

We also see the dangers of advanced digital networks and AI, which lie in the fact 
that they create new types of long-term, profound, and unpredictable dependencies for 
individuals, institutions, and states, which only a few can escape. In this context, we dis- 
tinguish between opportunities and risks that arise from different social embeddings of 
technology use and technology-immanent risks that always arise from the development 
and use of technology regardless of this embedding, and these technics-intrinsic risks 
must also be comprehensively addressed. 
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The latter includes, among other things, consequences of the daily amount, we are 
using digital technology to improve our fitness and health, in our social relations, or in 
education. But there are also technique-intrinsic risks for our national security. Tech- 
nics-intrinsic risks include, for example, hidden manipulations through the processing 
and utilisation of unmanageably large amounts of data (big data), which are used by 
companies to control consumption, by political groups for disinformation, and by au- 
thoritarian states to control their citizens, but also by (il-)liberal groups in western 
democracies for social control. 

Further digitalisation will only lead to humane and socially desirable outcomes if 
future decisions serve our common interests as humankind to a greater extent than in 
the past. 

This introduction already shows the necessity of a scientifically sound value 
framework for meaningful discussions on how possible influences of digitalisation 
(on people, societies, and the environment) can be made visible and assessable and, 
above all, what standards we apply. Measuring and evaluating is not meant in purely 
quantitative terms but includes, over all, a qualitatively accurate approach to determine 
the object. Measuring and evaluating, however, includes the exact determination of 
which parameters determine the result of digital computing processes as well, and 
which different “real-world” effects the feeding of certain parameters has (cf. Becker, 
2015, pp. 91-97). 

This compendium aims at contributing to a rational discourse by providing basic 
knowledge about the latest trends, impacts, and possible consequences in relevant areas 
of digital interconnectedness and artificial intelligence. 

To this end, the first part, “Humans and Digital Technology”, provides basic know- 
ledge and poses important philosophical questions about the relationship between 
humans and technology. The core question is whether digital technology still has the 
character of a man-made tool or whether machines and algorithms are increasingly 
being awarded an intrinsic value, and people are granting digital technology strong 
influence to the point of de facto domination over their everyday lives as well as essen- 
tial areas of society. The amount and intensity of smartphone and social network use 
initially suggest the latter. The second part of the book therefore deals with the need for 
legal regulation, be it “hard law” (e.g. laws and sanction-proof international agreements) 
or “soft law” (e.g. norms and technical standards, voluntary commitments). The third 
part of the book justifies for six central social areas why future steps of digital networks 
and AI development must be subjected to a comprehensive technology assessment, for 
which we formulate requirements in the fourth part. 

Thus, the book is divided into four parts: 


Part I: Humans and Digital Technology, 

Part II: Necessity of Legal Design, 

Part III: Political Shaping of Digitalisation, and 
Part IV: Responsibility of Science 
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The World We Want to Live In 


focus: 


The first three parts each provide different perspectives on the respective thematic 


In Part I, we look at the nature and perception of data, as well as the current state 
of developments in information technology, mathematics, and physics (Chapters 
1 and 2). Building on this, we discuss path dependencies in a mathematical-soci- 
ological analysis (Chapter 3) and deal with technology philosophy, the image of 
humanity, human enhancement, and Trans-/Posthumanism (Chapters 4 and 5). 
Part II shows the need for and the possibilities of regulation with a view to emerg- 
ing demands for machine rights (Chapter 6); questions of liability (Chapter 7); 
technical standardisation (Chapter 8) and intellectual property rights (Chap- 
ter 9). Of particular importance are the questions of international regulation 
of lethal autonomous weapon systems, as an example of future digital warfare 
(Chapter 10). 

Part III highlights socio-economic, cultural, and political issues of application 
and focuses on those areas of application that we consider particularly impor- 
tant: education (Chapter 11), health (Chapter 12), sustainability (Chapter 13), 
economy (Chapter 14), labour (Chapter 15), and social systems (Chapter 16). 


Throughout the 16 chapters, a chain of reasoning emerges, which we condense and 
conclude in the fourth part: “Responsibility of Science” with our demand for compre- 
hensive technology assessments of digitalisation, networking and artificial intelligence. 
I owe my thanks to Fatih Birol, Dr. Miriam Engel, Dr. Ginelle Greene-Dewasmes, 
Astrid Jekat, Dr. Peter Michael Link and Dr. Kuma Sumathipala for helping us in trans- 
lating the book into English! 

The end of our introduction is the beginning of our hope that this will be an exciting 
reading experience for you and we invite you to reflect with each other and with us, the 
Federation of German Scientists VDW, on the best way forward! 
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Part | 
Humans and Digital Technology 


Introduction 
Klaus Peter Kratzer 


“We can only see a short distance ahead, but we can see plenty there that needs to be 
done” (Turing 1950) - once again, we are confronted with this last sentence taken from 
Alan Turing’s fundamental publication on the question of whether machines can think. 
The difference to Turing’s situation more than 70 years ago, however, is that we are not 
only cultivating remote visions, but that we are in the midst ofa transformation of our 
living environment, in which machines exert an increasing influence on usin every way 
- we communicate with machines, use machines as mediators (in form and content) of 
our communication with fellow human beings and allow machines to classify, evaluate, 
and make judgments on us. The fact that the latter usually happens outside of our per- 
ception and often without our direct knowledge is irrelevant - we often see ourselves in 
the grip of an inevitability that is restricting our ability to choose and, driven by a desire 
towards conformity, is impairing a rational understanding of the situation. 

What is required of us is a desperate, blind faith in technological progress, because 
only very few of those affected can assess the state of the art, the reliability of this 
technology, and even fewer can grasp and evaluate the projected state of our society in 
future. In addition, of course, there is the question of who invests in such a technology: 
Which business model of which organisation, which company will benefit, and in which 
role does each individual find himself or herself? Is it inevitable that the manufacturer 
of my car is recording and analyzing my movement patterns? Do I have to put up with 
being schematically bossed around by a “chatbot” when making a complaint? Is it per- 
missible for my employer to access data on my body functions via a fitness wristband 
(provided by the company, of course)? ... Of course, I had already agreed in advance to 
the inevitable: to wear this wristband on the company premises at all times. 

The chapters in the first part of this book confront us all with the state of the art and 
help us to give substance to the reflection and debate that is still ahead of us. Gaining 
deeper insights, more questions will be raised than answered, as, lamentably, there are 
no simple solutions to complex problems. Nevertheless, we will all have to resolve these 
questions in a timely manner, so that as many options as possible for our future world 
remain open. 

The first chapter, Datafication, Disciplining, Demystification by Stefan Ullrich, tells 
the story of the centuries-long tradition of recording, coding, and structuring data, 
based on technical development, but also, and especially, on the complex relationship 
between data, facts, and the real world. 

In the subsequent chapter, Alexander von Gernler and Klaus Peter Kratzer introduce 
the technology underlying networking, digitalisation, and artificial intelligence in the 
chapter Technical Foundations and Mathematical-Physical Limits. They show where 
the possibilities and limits of this technology lie today, and, in consequence, what the 
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foreseeable future is holding in store for our society as a whole, but also for each of its 
members. 

In her contribution on Path Dependency and Lock-in, Jasmin S. A. Link shows how 
the socioeconomic dependency on digitalisation and networking, which is already 
widely observable, can lead to complete dependency and loss of freedom within the 
system logic by exponential amplifying effects, which can only be countered by an 
extensive system break. She argues for situational identification and development of 
alternatives and diversity as a counterweight to homogenisation based on algorithms. 

Stefan Bauberger looks at the relationship between technology and humans in his 
contribution Questions in the Philosophy of Technology. Humanity is in danger of being 
debased, even degraded, into the role ofan information-processing system. Technology 
must not be an end in itself: a differentiated, value-based consideration of technology 
must be achieved; what is technically possible is not necessarily what is permitted or 
required. 

Following on from this, Frank Schmiedchen takes a look at interfaces between man 
and machine and utopian/dystopian visions ofthe future ofthe anthropopocene in his 
chapter on Digital Extensions of Man, Transhumanism, and Posthumanism. While the 
evolutionary advancement or replacement of Homo Sapiens by algorithms seems to 
be a quasireligious (nightmare) dream in the distant future, digital “enhancements” in 
the guise of clothing, jewellery, and implants are already commonplace - the symbiotic 
relationship of many people to their mobile phones, which is nowadays quite common- 
place. supports this vision. 

These chapters will lead to reflexion, thought, and discourse, surely also to friction. 
But when we enter such a discourse and when we can feel the heat of friction, an im- 
portant goal of this section, better even, of the entire book, has already been achieved. 
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Chapter 1 


Datafication, Disciplining, Demystification 
Stefan Ullrich 


Data, that what is given - if Francis Bacon has his way (Klein and Giglioni, 2020). Data, 
that what has to be regulated - ifthe European Commission has its way (EUCOM, 
2020a). In between are 400 years of collecting, storing, processing, and disseminating 
data. While for Bacon it was self-evident that data primarily served science, today we 
speak of data as a commodity, which is said to be worth over 325 billion euros in Europe 
alone (EUCOM, 2020b, p. 31). Etymologically, then, the Latin datum and commodi- 
tas, the given here, and the useful, suitable, advantageous there, are approaching each 
other. This should make the scientific mind sit up and take notice, because whenever 
something is too suitable for one’s own theory, it is necessary to take a closer look. Data 
should be uncomfortable, should be able to challenge one’s own theory (and, if we cheat 
a little like Mendel with pea cultivation, sometimes also confirm theories). 

This year we celebrate the 75th birthday of the Electronic Numerical Integrator and 
Computer, or ENIAC for short, the first freely programmable, electronic universal com- 
puter. Sure, in Germany we could also celebrate the 80th birthday of Konrad Zuse’s Z3 
and in the UK the 85th birthday of the Turing machine - in short: we have been living 
in an age of computation using universal computers for several generations. Data have 
been seen for decades as machine-readable and, above all, computable information. The 
word information, that which is put into form, reveals the process of transforming the 
pure notation of numerical values based on observation or deliberation into structured 
formats. Data are more than numbers or symbols; they have a scheme, have been mod- 
elled and prepared in a machine-readable way. 

A very simple - and yet ingenious - scheme is the table. The first row contains des- 
ignations, such as measured quantities and units, while the other rows contain symbols, 
written down in pictures, letters, and numbers. Leibniz described the power of the table 
to his sovereign in flowery words. The busy mind of the ruling person could not possibly 
know how much woolen cloth was being manufactured in which factories and in what 
quantity was demanded by whom in the population. However, since knowledge of this 
“connexion of things” was indispensable for good government, Leibniz proposed so- 
called state tables (“Staatstafeln”), which make complex facts comprehensible at a glance 
and thus governable and controllable (Leibniz, 1685). 

Data serve to control the human being. Initially, this is only to be understood as 
genetivus subjectivus, i.e. humans use data to control their environment. Recently, 
however, the meaning in genetivus objectivus has also been debated: Data are used to 
control people. In this paper, we start with the building blocks of data as we understand 
it in the modern context, so we begin with the machines that process data. Then we look 
at the two dimensions of data for human control. 
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1.1 Building Block: Digital Number 


For modern data understanding, the number is probably the most important building 
block. With its mantra “Everything is digital number”, computer science, in direct suc- 
cession to the Pythagorean school of thought, advocates the view that every gesture, 
every speech, every image, every writing, in short, all codified human actions can be 
written down with the help of a number (cf. Ullrich, 2019). This is not true, of course; 
the most important things cannot be recorded, such as what constitutes a good-night 
kiss, in essence. Poetry comes closest to this, but even the best poems fail at a high level 
to capture the innermost part ofthe human being. But whereof computer science cannot 
speak, thereof it does not keep silent about, but collects data. The number of goodnight 
kisses correlates with the size of the family or shared apartment, an important data point 
for software rights holders who want to sell licenses for use. 

The number, or more precisely, the discrete number breaks down the intangible 
continuum of the environment around us into measurable and countable objects; the 
measurements and numbers give us a sense of control. It must have been extremely 
reassuring for the first communities to unravel the mystery of the seasons. That winter 
just doesn't last forever, but is replaced by spring! Eight hours of daylight at the winter 
solstice, a good six lunar phases later, it is already sixteen hours - with the help of the 
calendar and a look at the date, man at the mercy of the weather has little control. Even 
today, in the age of man-made climate change, it is data that support our climate models 
and tell us about our future. 

The discrete number served above all the empirical sciences, produced with the 
help of observation or with the help of instruments and tools. In the mechanical age, 
tools such as the telescope provided analogue signals that first had to be schematised 
or immediately discretised - Galileo drew the moon with its less than perfect surface 
schematically, and to this day mushroom pickers and physicians consider the schematic 
representation of fruiting bodies or nerve cells more didactically valuable than high-res- 
olution photography. 

To convert an analogue signal into a discrete one, an analogue-to-digital converter 
is needed. The continuous signal, such as a sound wave, is measured 44,100 times per 
second, or “sampled” in technical jargon. The best way to imagine how this works is 
with the help of a grid. Imagine drawing a wave on a piece of graph paper. Then take a 
pencil of a different colour, for example red, and mark the intersections of the checks 
that are closest to the wave. These red points then mark the discrete values of the ana- 
logue signal (Figure 1.1). 

The resulting discrete, digital signal is, of course, only an approximation of the an- 
alogue signal, which is all the more congruent the higher the sampling frequency and 
the finer the quantification. The advantage is that we now have a machine-processable 
datum, a digital datum that we can store or communicate. 

The first analogue-digital converters were developed by Konrad Zuse between 1943 
and 1944 to mechanise the reading of the analogue dial gauges of the Henschel glide 
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Fig. 1.1: The red digital signal is the 
sampled and quantised representation 
of the grey analogue signal. (PD) Public 
Domain from Wikimedia Commons, the 
free media repository. 
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bomb Hs 293. Once the bomb had been dropped, it could be steered by radio thanks 
to ailerons and elevators to safely deliver its 300 kilograms of explosives to its target. It 
was the world’s most successful sea target missile, with “successful” translating to being 
responsible for the greatest US human loss in World War II. 

This drastic but quite typical example is intended to illustrate the role of data. Data 
are a demon that can be as subservient as it devours, as this book comprehensively 
shows. In the real world, there is no such thing as harmless data. Data have left the 
innocent sphere of mathematics and, since the invention of the punch card, determine 
the wealth and woe of people. 

In the punched card, data are encoded with the help of holes, and you don’t even 
need a machine to decipher them. In view of the enormous investment costs, the first 
programmers would not have been able to afford the luxury of using valuable machine 
time just for reading out the data, and even the execution of simple filter algorithms 
does not require a machine if the holes are suitably designed. Let us take an edge punch 
card, which, unlike other punch cards, is also suitable for manual processing. An edge 
punch card has holes all around the edge in the uncoded state. Now a key, a coding is 
designed, and the cards are notched so that a slot is created at certain points. In Figure 
1.2 we see both holes (as created with a hole punch) and slits (notches). All the cards are 
now placed on a stack and set up so that the holes are on top of each other. 

If you now push in a knitting needle and lift the stack of cards, those cards that 
have a slit where the needle is will fall down. The time saved in searching, in contrast 
to normal index cards, is enormous: the selection speed is between 30,000 and 40,000 
cards per hour. 

It is not the data analysis alone, but the very collection and categorisation of the data 
that promotes both its use and potentially misuse. With the help of an edge-hole map 
system, Leibniz’s state tables would have made a version leap, and who knows, perhaps 
they would have ensured that the sovereign would have been informed not only about 
the amount of wool, but about political adversaries, their personal data, habits, and 
meeting places. 
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Fig. 1.2: Manual selection of an edge punch card. Illustration taken from: Bourne, Charles: Methods 
of Information Handling, John Wiley & Sons, New York, 1963, p. 81. 


1.2 Data for Human Control 


What Leibniz’s state tables were to the sovereign, the smart fitness bracelets, and the 
even smarter universal computers in the trouser pocket are to the health-conscious 
person, which we still call a “phone” for historical reasons. We count steps, calories, or 
CO, emissions to better discipline ourselves. 

We want to control ourselves or our environment with the help of recorded data 
- but what does that actually mean? The contre-röle is the counter-register to confirm 
an assertion proven by data. Trust is good, control may be better, but to be effective it 
needs monitoring: we need to collect data again, this time independent from the data 
that went into the assertion to be proven (especially if the data come from others), and 
we need these data preferably in real time. 

This is the weak point of Leibniz’s state tables: the meaningfulness of the data nat- 
urally decreases over time. Sure, for monitoring one’s own actions, i.e. testing the effec- 
tiveness of political decisions, for example, annually collected data are sufficient, or, as 
Leibniz wrote: suitable for self-government. However, if we want to rule over someone, 
we need monitoring and data. We cannot meaningfully talk about current data-based 
business models or address data markets without explicitly mentioning surveillance, 
which is an instrument of human control. 

The icon of surveillance is, of course, Jeremy Bentham’s Panopticon, already briefly 
mentioned in the introduction to the book, his 1791 design for an “Inspection-House”. 
Bentham (2013) planned this for a variety of institutions, from schools to hospitals, 
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but the first and most famous example is the prison. At the centre of the institution is 
a tower that allows the cells radiating out to be seen, but protects the observer in the 
tower from being seen. Thus, the inmate of such a cell does not know ifand when she 
is being observed, but she knows that she can be observed at any time. This possibility 
of surveillance leads to a change in behaviour; the inmate permanently behaves as if 
she were actually being watched at all times. She has internalised the surveillance, this 
is what has become known as the “panoptic principle”. 

The pan-optical all-seeing eye becomes an all-seeing-for-ever-stored data based 
eye with the help of surveillance technologies. Just as the modern concept of privacy 
(“the right to be let alone”, Warren/Brandeis, 1890) only emerged ex negativo with the 
advent of the camera, the fundamental right to informational self-determination was 
only established with the advent of big data processing systems (BVerfG, 1983). Data 
protection, which is actually a very inappropriate term, picks up on the information flow 
direction of the panoptic principle. The informationally more powerful person watches 
over the informationally inferior occupant in the tower. Data protection, or more pre- 
cisely: data protection law, is now supposed to ensure that this power is not abused. 

But who monitors the monitors? The public, of course, the “body of the curious at 
large - the great open committee of the tribunal of the world”?, using publicly available 
data (Bentham, 1787). Protect private data, use public data, as stated in the Chaos 
Computer Club's Hacker Ethics, means to recognise the inherent nature of data and use 
them for the good of society (CCC, 1998). Data also serve to control people, sometimes 
understood as genitivus objectivus in the case of the Panopticon, sometimes understood 
as genitivus subjectivus in the case of the Court of Justice of the World. 

In biometric recognition systems, all the above-mentioned explanations come to- 
gether in a complex sociotechnical system. Biometrics, i.e. the measurement of life, is 
an instrument of statistics. Mortality tables, age structure of the population, and average 
life expectancy are of interest to state leaders when it comes to taxes, participation, and 
distributive justice. In one of the first scientific works on biometry, the Swiss natural 
scientist C. Bernoulli first describes how a table of life expectancy should be structured 
and what advantages arise from this clear connection of things, before pointing out 
in a somewhat hidden insertion that it was life insurance institutions that made the 
collection of these data “a necessity” (Bernoulli, 1841, pp. 398-399). Once the trans- 
disciplinary cultural technology researcher has found this technological history trail, 
she discovers the true motivations behind biometric systems everywhere. Since Francis 
Galton, dactyloscopy has not only served law enforcement purposes but also, like all 
other biometric measurement systems to this day, voluntarily or involuntarily, feeds 
racist mindsets and practices. 

Before we take this thought to its conclusion, the “black box” should be opened a 
little at this point. To do this, we will look at the typical structure of a system for the 
automated recognition of fingerprints (after Knaut, 2017, p. 44): 


2 Quoted after original source https://en.wikisource.org/wiki/Panopticon_or_the_Inspection-House 
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Fig. 1.3: Structure of a biometric system for the automated recognition of fingerprints. (cf. Knaut, 2017, 
p. 44) 


This diagram is already a considerable simplification of the actual architecture of a typ- 
ical system, which we can see from the “Fingerprint Acquisition” component alone. Before 
the development of the corresponding sensors, the fingerprint was captured in analogue 
form, as the name suggests. The print, typically after placing the finger first on an ink pad 
and then on a piece of cardboard, only leaves colour pigments where the papillary ridge 
of the finger is located, precisely these typical lines that we also see with the naked eye. 

Biometric recognition systems are used for verification and identification and are 
usually marketed as access systems (verification) or generally as official security technol- 
ogy (identification). The introduction of biometric passports and ID cards in Germany 
was also presented from this point of view. However, in background discussions and 
when asked directly, it is clear to all involved that this is a business promotion, as the 
corresponding readers have to be licensed. However, the data-based business models 
of biometric recognition systems have a catch: They technically fall under the General 
Data Protection Regulation (Article 9 (1) GDPR), which makes exploitation so chal- 
lenging. Biometric data are also the most intimate and visible data: Unless there is a 
pandemic, we are constantly showing our face. And even in Corona times, our walk 
in a crowd of people can be quite unique. Finally, there are our fingerprints, which are 
emblematic of identity, although technicians and scientists have been pointing out for 
decades that identity constructions and attributions are at stake. However, data can also 
be used to question these attributions, which is what the last section is about. 


1.3 Data for Demystification 


Data are the key to knowledge; they are the basis of the empirical sciences and provide 
a view of the world not only for quantitative but also for qualitative researchers. Data 
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are not facts, which was important to Francis Bacon, and we should always bear that 
in mind. Data can generate, confirm, or question facts in the scientific mind. Data can 
also obscure facts. Data science is slowly maturing into the basic cultural technique of 
the responsible member ofthe networked society. Data scientist Hans Rosling demon- 
strated to a large audience (and thanks to audiovisual data also on Youtube, Vimeo, and 
others) how data can be used to bridge cultural differences, break down prejudices, and 
ensure common understanding. In a very humorous and exposing way, Rosling holds 
up the mirror to us that we rely on data, numbers, and facts that we learned in school 
and that are now reproduced on all media channels. Our conceptions of countries in 
the global South, for example, are closer to myth than to the present (Rosling, 2006). 
Demystifying false, perhaps even harmful assumptions with the help of data was the 
main drive of the humanist Rosling. 

But for this to happen, these data must also be available. There is a fundamental bias 
when it comes to data: we can only measure what is measurable. So, on the one hand, 
this depends on instruments and tools, but on the other hand it also depends on culture 
and customs. It is not due to a lack of tools that Caroline Criado Perez (2019) observed 
a gender data gap, but due to culture of a male dominated society. Data are collected for 
a purpose, and the more effort put into data collection, the more likely one is to expect 
a dividend: Data thus becomes a means of payment. 

Data are the central element of digitalisation because it comes from both the old 
world of automated data processing and the new world of heuristic data techniques such 
as machine learning, big data and artificial intelligence. It is therefore not surprising that 
the development of data literacy is repeatedly insisted on, without, of course, saying 
what exactly it should look like. Demystification also includes a sobering look at current 
data processing practices. The majority of people usually choose not to deal extensively 
with data, not even with the data they generate themselves, which becomes usable for 
whatever purpose by consenting to unread declarations of use by “someone” In a free 
society based on the division of labour, we should also accept this, but then legislators 
must hold computer scientists and companies with data-based business models more 
accountable, for example by demanding that data-based business models not be subject 
to any secrecy obligation or that the data-processing systems be precisely labelled. 

Another step towards demystification could be didactic systems like MENACE (cf. 
Ullrich 2019c). MENACE was the name of a didactic machine to teach machine learning 
principles that was conceived and described by Donald Michie in the 1960s. His ma- 
chine could play Noughts and Crosses (also known as Tic-Tac-Toe, Three in a Line, or 
Tateti) against a human player (cf. Michie, 1961). The Machine Educable Noughts And 
Crosses Engine was a machine learning system, but with a special feature: the machine 
was not made of computer components, but of matchboxes filled with coloured beads. 
Each colour represents one of the nine possible positions that an X or an O can take on 
the playing field. The construction was simple and impressive at the same time; no less 
than 304 boxes were needed for it, one box for each possible configuration in the game. 
The operator now randomly draws a coloured pearl from the respective box with the 
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corresponding configuration. In the course ofthe first few games, the matchbox machine 
will probably lose, as there is no strategy at all, since the beads are drawn randomly. 
But then machine learning kicks in: If MENACE loses, all the drawn beads that led to 
the defeat are removed. If MENACE wins, three beads of that colour are added to the 
boxes used. This means that the chance of losing is reduced, while on the other hand 
good moves are significantly rewarded. If MENACE is trained long enough, it “learns” 
a winning strategy (by improving the chances of making good moves) and, therefore, 
“plays” quite well. 

The interesting thing is that no human would attribute any intention to a stack 
of boxes, unlike machine learning systems implemented with software on a computer 
hardware. Especially in the case of machine learning or artificial intelligence, as a critical 
observer of the information society, one can still marvel at “the enormously exaggerated 
attributions an even well-educated audience is capable of making, even strives to make, 
to a technology it does not understand” (Weizenbaum 1976, p. 7). 

Everything starts with the will to understand, in order to be able to use the power 
of data accordingly for the benefit of the general public. The Federation of German 
Scientists is aware of this special responsibility that the influence of technological-sci- 
entific progress has on people’s mindsets, and that is why we are committed to regaining 
informational sovereignty as a networked society as a whole. 
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Technical Foundations and Mathematical-Physical Limits 
Alexander von Gernler and Klaus Peter Kratzer 


Acknowledgements: Astrid Jekat helped us a lot when translating our chapter to English by 
giving us many valuable comments and insights as native speaker that greatly improved 
the quality our our text. 


To discuss the basics used in this book, we chose a slightly different order than in the 
title of the book. The three terms networking, digitalisation, and artificial intelligence 
covered here build on each other and are more easily introduced that way. 


2.1 Networking 


Networking is the foundation of all topics discussed in this chapter. It describes the 
interconnection of computing or storage units, as well as sensors or actuators, by means 
of any transmission medium for the purpose of information transfer. The distances 
bridged can be as short as a few centimetres (near field communication, NFC) or as 
long as several thousand kilometres (transatlantic Internet cable). 


[STS E 


Fig. 2.1: Examples of connectivity using general graphs: A weakly connected and disconnected 
graph, a connected graph, a fully connected graph. (Source: The authors) 


The degree of interconnection can be explained using Figure 2.1: The more edges there 
are between the nodes in a networking scenario, the more possible paths can be taken 
between two specific nodes. Accordingly, the availability of individual nodes also in- 
creases - improved interconnections make the network robust against failures. 
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2.1.1 Technical Aspects 
2.1.1.1 History 


Beginning with the legendary and world’s first connection of the first four computers via 
long-distance links in the so-called ARPANET in 1969 (Figure 2.2 shows a more devel- 
oped version from 1973) in the USA, the development of ever faster and more robust 
transmission technologies for information transmission continues unabated. Important 
parameters for characterising a networking technology are bandwidth? , latency, carrier 
medium, and range. 

To this day, the transmission media used are almost exclusively copper wires, elec- 
tromagnetic waves (radio technology), or fibre optic media‘. 


Fig. 2.2: Map of the ARPA network in 1973. (Source: ARPANET) 


3 Often loosely denoted as line speed. 
In exceptional cases such as hacking or activity of intelligence agencies, alternative media types like 
local sonic waves or significant variations of computer power intake can be employed to stealthily mod- 
ulate and transfer information between otherwise non-connected devices. When used in this manner, 
they are often referred to as side channels. For practical everyday information transfer, however, their 
significance is negligible. 
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2.1.1.2 Modelling 


To apply information to an otherwise raw transmission medium, so-called logical layer 
models are used in both telecommunication and information technology. By modelling 
in different layers, it is possible to realise, among other things, different technical re- 
quirements such as a) protection against data loss on the transmission path, b) security 
properties such as encryption ofthe communication, c) customer needs such as pro- 
viding several virtual channels on a single physical line, or d) performance properties 
such as guaranteeing a certain bandwidth or latency. 

An essential model in computer science in the context of networking is the ISO/OSI 
layer model (Tanenbaum (2012), Figure 2.3). It allows experts to discuss different aspects 
of data transmission at respectively assigned levels of meaning. In this text, the layer 
model is intended only as an overview of the terminology used in computer science. 


abstract | 7 application i.e. HTTP 
6 presentation 
5 session 
4 transport i. e. TCP 
3 network i. e. IP 
2 data link 
concrete | 1 physical i. e. copper wire, fibre optics, wireless 


Fig. 2.3: The ISO/OSI layer model used in computer science with a few examples of selected abstrac- 
tion layers. By means of such models, complex communication issues can be individually discussed 
based on different concerns. (Source: The authors) 


2.1.1.3 Unit of Information: Bit 


The information content of a message is measured in bits”. A bit can only have the value 
0 (negative, false) or 1 (positive, true). The unambiguous answer to a yes/no question 
can be encoded by means of a single bit. A bit is therefore the smallest possible unit of 
information. 

For the unambiguous coding of the four cardinal directions north, south, east, and 
west, on the other hand, two bits are necessary: One bit can be used, for example, to 
state whether the cardinal direction lies on the north-south or on the east-west axis, 
the other to state whether one or the other direction is meant from the selected axis. To 
extend the representation to north-east, north-west, south-east, and south-west, a third 
bit is necessary (Figure 2.4). 


5 This definition goes back to Claude Shannon and his fundamentally important Theory of Information 
(Shannon 1948). 
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positive or 

negative? 
horizontal rotate by 
or vertical? 45 degrees? 


0 Fo Ra eee north 


--- north-east 
1 of ol SSS sives west 


--- north-west 
1 [afol en east 


--- south-east 


Fig. 2.4: Table of the eight cardinal directions with coding as a sequence of 3 bits. (Source: The authors) 


A systematic approach is recognisable in these examples: with a given number of n bits, 
a certain amount of different N of different possibilities can be coded. The connection 
here is N = 2”. The number of possibilities thus increases exponentially with the number 
of bits of information, virtually exploding! 256 bits already allow such an enormous 
number of possibilities that the resulting power of 2°% exceeds the number of atoms on 
earth assumed by science of about 10°° by a multitude of orders of magnitude’®. 


2.1.1.4 Entropy and Compression 


However, not every data stream of the same length carries the same amount of informa- 
tion. Information also corresponds to the degree of surprise that viewers are exposed to 
when the next fragment of information arrives (Figure 2.5). 

Data with low entropy hardly cause any surprises for the observer. The lowest pos- 
sible entropy thus is expressed by chain of bits (a so-called bitstring) whose bits all 
have the same value. A bitstring with very high entropy is hardly distinguishable from 
so-called white noise, i.e. total randomness: Each additional bit has either the value 0 
or 1 with the probability p = 0,5. 

Data packets with low entropy, such as text files, can be compressed into smaller packets 
with high entropy using compression algorithms. In Figures 2.6, 2.7, and 2.8, the image 


6 Consider the story of the chessboard: Place one grain of rice on the first square, two on the second one, 
then four, then eight, and so on. The amount of rice in the entire world will not be sufficient to fill the 
chessboard up to the last square when applying this method. 
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41444394434 4 


Fig. 2.5: Example bit chains with minimum or high entropy, as well as expected continuations by ob- 
servers. (Source: The authors) 


Figs. 2.6, 2.7, and 2.8: Example images with minimum (left), low (middle), and high entropy (right). 
(Source: The authors) 


with minimal entropy will hardly surprise the viewer: It is entirely filled in one colour 
and can thus be described exactly in a few words. The middle picture, however, already 
is so irregular from a human point of view that it can only be roughly, but not exactly, 
described by natural language. And the third image shows so-called white noise, i.e. the 
presence of the highest entropy: in contrast to the middle image, each pixel is chosen 
completely at random. 

The degree of compression depends on the algorithm used and corresponds to the 
achieved approximation of the entropy of the final product to white noise. Optimal 
compression can be achieved by using an ideal compression algorithm such as Huffman 
coding (Huffman, 1952). In this case, the data are reduced as best as possible to the 
size of its actual information content. Compression is often applied to large amounts 
of low entropy data to reduce the transmission time required within a communication 
network. Examples include image and video signals (for which even more specialised 
algorithms exist), but also software updates. 


2.1.1.5 Development of Networking 


« Bandwidth 


Transmission bandwidth has been subject to rapid growth since the 1970s (Figure 2.9), 
bridging several orders of magnitude in just a few decades. While initially the reliable 
transmission of a few kilobits per second (kbps) was considered a great success, band- 
widths soon rose into the range of megabits (Mbps), gigabits (Gbps), or in the backbone 
area even terabits (Tbps). 
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Fig. 2.9: Chart showing the development of Ethernet transmission speeds. While time progresses 
linearly in the x-direction, a logarithmic scale in the y-direction is necessary for the simultaneous repre- 
sentation of the speeds of all technologies since 1980. (Source: Ethernet Alliance, Ethernet Roadmap 
2020) 


At the time of printing of this book, up to 100 GBit/s is a still expensive but feasible 
type of networking in a professional environment, for example to connect parts of a 
building on a company or university campus. However, networking with 1 GBit/s is 
much more common, for example for office networks but also for private applications 
within one’s own household. 

Wide-area network uplinks, such as the home Internet connection for private users, 
currently range between 50 and 200 MBit/s, with exceptions at the bottom and at the 
top. They often represent a bottleneck in the population’s access to the Internet. Due to 
the social importance of Internet access (participation in social life), there is a growing 
view that the state is obliged to provide the population with fast connectivity nation- 
wide. One keyword here is the German Federal Government’s broadband initiative. 


e Latency 


But pure bandwidth is not everything: If, for example, the transmission of data packets 
between two computers were to take several seconds, the affected line would be com- 
pletely unusable for Internet surfing. Also, the so-called real-time capability of sensors 
and actuators requires them to be connected via a low-latency connection. Usually, 
acceptable latencies of today’s Internet services or news sites are in the range of 20 to 
100 milliseconds (ms), i.e. 20-10% “s”. 

The latency of transmission has continuously dropped since the 1980s. In the mean- 
time, however, this variable has largely reached optimised values, since the speed of 
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information is practically always’ limited by the maximum propagation speed of waves 
in the respective carrier medium. This in turn is limited by the speed of light in the 
medium, and ultimately by the speed oflight in a vacuum, a natural constant called c. 


« Energy Efficiency 


With each development step towards more bandwidth, the efficiency of energy trans- 
mission has also increased: The energy required to transmit one bit ofinformation over 
the same distance has been continuously reduced. Here too, there is an absolute lower 
limit based on Planck’s constant: Information processing and transmission cannot be 
done for free, but are always accompanied by a change of energy levels in the media 
or transmitters involved. And these energies are quantised, i.e. they follow discrete, 
indivisible steps. Therefore, when a bit of information is transmitted, it is theoretically 
impossible to fall below a certain minimum quantum of energy expended. In practice, 
however, mankind is still very far from this lower limit as a large number of electrons 
or photons are used for reliable transmission instead of only a single one. 

But even for this extent, quantisation still applies. This is one of the reasons why 
IT can have a paradoxical effect: On the one hand, it can be ostensibly energy-saving 
and sustainable (paperless office, avoidance of travel). On the other hand, IT builds 
up a foundation of primary energy consumption that stems solely from the provision 
of fast transmission paths. The video conferences that have become so popular in the 
Covid Age, but also the streaming of films, have a strong negative climate effect that also 
depends on the transmission medium employed (BMU, 2020). 


e Bridged Distance 


Depending on the type of connection purpose, the literature refers to certain classes of 
communication networks. Therefore, different technologies are used for their respective 
implementation (Table 2.1). 


Tab. 2.1: Transmission distances of different media 


Medium Typical distance 

Near Field Communication (NFC) 10cm 

Bluetooth 10m 

Wireless LAN 38m 

Ethernet (copper) 100 m 

Fibre optics up to 100 km, with repeaters even thousands 
ofkm 


7 By exploiting certain physical effects using quantum entanglement, transferring information in ze- 
ro-time appears to be possible. However, this kind of zero-time transfer is currently only available in a 
lab environment, and will thus be of no practical relevance for the conceivable future. 
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2.1.2 Social Aspects 


2.1.2.1 Widening ofthe Focus 


Through networking, people with the same interests are finding together who would 
not have been able to do so without digital networking, for example because they live 
on different continents or because it is difficult to communicate their interests. For 
many unusual activities, hobbies, interests, and ways of thinking, communities are now 
being formed that are no longer tied to a specific location, as was the case in human 
associations mere decades ago. 

The other side of the coin is that the filtering levels of the old world that were con- 
sidered meaningful, such as editorial offices, proof readers, and publishers (and many 
other such instances in our society), are now losing or have completely lost their im- 
portance. This has not only resulted in the advance of so-called transparency and im- 
mediacy, or even the disappearance of what is perceived as censorship, but also ensures 
that potentially all news and messages can reach all recipients in an unfiltered manner. 
In reality, the filtering of messages has only shifted to the recipient. On the recipient 
side, however, there are now several possibilities: Filtering a) can ideally be carried out 
in full, but costs additional, unplanned time, b) cannot be carried out due to lack of 
competence, c) is omitted because of excessive demand. 


2.1.2.2 Creation of Parallel Worlds 


The relevant social media platforms with business models based on maximising atten- 
tion have once again reinforced this trend and brought it to life in a new quality. The 
filtering mentioned above, which was previously shifted to the user, is now taking place 
again. However, this time the filtering is not carried out in the interest of the user, but 
following the business interests of the platform. Primarily news that generates attention 
is disseminated, regardless of its actual truth content (fake news) or its social appro- 
priateness (incitement, defamation, hate speech). The generation of rapidly spreading 
so-called Internet memes is also observed, i.e. minimised information such as slogans, 
images, or short videos. Politically, these platforms have also contributed to the rise of 
populist movements in many countries around the world. 


2.1.2.3 Social Isolation 


The business model is also based on the man-in-the-middle principle: the platforms 
can only be successful ifthey control, evaluate, and influence the information flowing 
between the participants. It follows that the platform operators have an inherent in- 
terest in the isolation of the participants from each other: All communication - even 
between acquaintances who are networked locally - should exclusively take place via 
the platform. This isolation is presented to the platform users as attractive through 
psychological stimuli (such as instant gratification or fear of missing out). 
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In his book “Ten Arguments for Deleting Your Social Media Accounts Right Now”, 
Jaron Lanier (Lanier, 2018) describes the social consequences of this intention to iso- 
late. The core message here is: hate sells, i.e. sensational news (hate and catastrophes, 
exaggerated or untrue news) spreads much better than “boring” (but true or good) 
news (Dizikes, 2018). The business models of the platforms mentioned above aim to 
maximise the attention of visitors to their own site. This can be achieved by displaying 
news items that generate a high response from the audience, regardless ofthe content. 


2.1.2.4 Shifting of Metrics Previously Assumed to Be Stable 


Networking is also shaking up previous certainties. Such shifts are not new and happen 
in the course of every innovation. In the 1980s, for example, a grandmother would ask 
her grandchildren on the phone to keep the conversation short, because long-distance 
calls were so expensive. That is long over. 

However, with digital networking, certainties are being overturned at a much faster 
rate than before. One such belief has always been that local is synonymous with fast 
or cost-effective. Since the existence of broadband connections, this no longer unre- 
strictedly applies: for example, most people would now rather type a search term into 
their tablet on their couch than look for the answer in a book on their bookshelf, even 
if the local proximity of the book is only two metres as opposed to several hundred 
kilometres. This shift in metrics makes new business models attractive that previously 
were not realistic. The cloud in its entirety, but also the streaming of music and video 
are among them. 


2.2 Digitalisation 


2.2.1 Definition 


Digitalisation is the progressing spread of various types of digital technology and its 
increasing penetration of all aspects of social, economic, and political life, combined 
with an increasing digital representation of analogue processes in the real world and the 
evaluation and use of the data this generates. 


2.2.2 Motivation 


This effort is not an end to itself. Especially in private economy, digitalisation is the 
result of permanent optimisation efforts: The more transparent an organisation’s pro- 
cesses, inventories, orders, customer relationships and many other such parameters are, 
the more efficiently it can plan, and the more informed strategic decisions it can conse- 
quently make. Digitalisation also makes new business models possible in the first place. 

Of course, the digitalising organisation is in competition with other organisations 
that are also trying to leverage these advantages. 
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2.2.3 Interaction of Digital and Real World 


Asa partial aspect of digitalisation in industry, the German National Academy of Science 
and Engineering (acatech) created the concept of cyber-physical systems (Geisberger 
and Broy, 2012). It refers to Norbert Wiener’s cybernetics and describes an important 
point in digitalisation: computing systems no longer operate only on information in 
data storage but also exchange data with the real world by measuring and manipulating 
it using sensors and actuators. 

Science fiction literature had already invented the word cyberspace for the global 
digital space by the 1980s, i.e. without knowing about today’s Internet (Gibson, 1984). 
Today, the term is also occasionally used synonymously for the Internet, especially in 
a popular context. 


2.2.4 Pervasion 


The digital world is thus increasingly penetrating the analogue. In the early days of 
information technology, the focus was on well-structured application fields with clearly 
defined data collection and processing rules (e.g., payroll accounting, merchandise 
management systems). Now digitalisation is penetrating application scenarios with 
fuzzy and multidimensional assessments and forecasts as well as complex action spaces 
with a high number of degrees of freedom. A brief example for the latter: While the gui- 
dance of an underground railway with its stringent rail guidance and relatively simple 
operational organisation is quite controllable algorithmically, and therefore driverless 
trains are already in operation in many places, the driverless, autonomous guidance ofa 
motor vehicle in open road traffic is still technically extremely difficult and dangerously 
error prone. This is due to the many times higher number of degrees of freedom in 
guiding a motor vehicle in road traffic, which cannot be safely controlled with current 
technology. Nevertheless, vehicles from a certain manufacturer with a beta version of a 
Level 5 autonomous driving programme are currently driving in the USA in an unreg- 
ulated manner. The lack of humility visible here when facing the complexity of reality 
combined with naive faith in technology is highly questionable and needs to be fully 
investigated and discussed. 

It turns a previously highly structured application environment into an increasingly 
unstructured one with many parameters. New environments invaded by digitalisation 
are increasingly complex. Computers are given more “responsibility” but, being ma- 
chines, cannot logically gauge the effect of their actions, especially the ethical dimension. 

As increasing measurement data are now available due to the penetration of digital 
technology into previously untouched areas, more and more everyday and previously 
insignificant actions and facts can be linked and analysed in new contexts. For example, 
it is now possible to count the steps of users by means of the movement sensors in smart- 
watches or smartphones, or even to obtain vital data via other sensors. From a technical 
point of view, nothing stands in the way of correlating data obtained this way with data 
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from completely different areas of life (such as account balances, shopping behaviour, 
GPS movement patterns, call behaviour). Very easily available and apparently irrelevant 
features such as first names, addresses, and shopping habits can be used to assess the 
creditworthiness of the persons concerned (Leeb and Steinlechner, 2014). 

Another example: There appear to be apps that can detect Parkinson's disease by 
collecting data on smartphones (EU COM, 2020). 


2.2.5 Causality and Correlation 


However, as not every coincidentally discovered correlation of data points must point 
to real causality®, the users observed in this way are always in danger of having a certain 
behaviour or causality attributed to them by the data collecting parties without their 
knowledge. 

This also includes the danger of being treated according to categorisations made 
(e.g., not creditworthy, alcoholic, irregular lifestyle) in the next interactions with digital 
systems. In contrast to the principle of the rule of law, which would make the accused 
aware of the accusation made against them and hear them out on the matter presented, 
here the persons concerned are not informed about the modelling, i.e. the making of 
assumptions, that takes place in the background. And this, of course, automatically elim- 
inates the opportunity for those affected to object or make a counter statement. Despite 
all the advantages, digitalisation also makes it possible to charge previously insignificant 
or meaningless facts and actions with (sometimes only seeming) significance simply 
because they can now be measured. 


2.2.6 Digital Persona 


The representation of the individual through the totality of his or her data is often re- 
ferred to as the digital footprint. The danger outlined above that individuals are judged 
by their digital representation rather than by their real person or their real actions 
has a social impact that fits well into the techno-optimistic-utilitarian zeitgeist, aka the 
self-optimisation of individuals. Just as the stigmatisation of people by algorithms is pos- 
sible through the Digital Persona, so is their preferential treatment through the portrayal 
of a best possible digital profile optimised according to criteria of social desirability. 


2.2.7 Social Consequences 


This influence of the digital world on the decisions of individuals also results in social 
pressure: for example, in the terror hysteria that followed the attacks of September 11, 
2001, page visits of “suspicious” articles with keywords such as bomb on Wikipedia fell 
significantly and disproportionately. Science explains these so-called chilling effects with 


8 An excellent visualisation of this principle is offered by the Website ‚Spurious Correlations’ (Vigen, n.d.) 
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peoples’ feeling of being watched. This feeling promotes the display of socially desirable 
behaviour, or behaviour that the individuals concerned consider to be socially desirable 
(Assion, 2014). 

Closely related to chilling effects is the concept ofthe panopticon brought into play 
by Jeremy Bentham (Bentham, 1995). Originally conceived as an innovation for the 
penal system in Great Britain, the panopticon is now also used synonymously for the 
possibility of being observed at any time. 

Going further, the knowledge of this omnipresence can be elegantly exploited with 
the help ofa so-called social scoring system (somewhat more euphemistically also: social 
credit system) (Everling, 2020). Such systems are widely used in the private sector vis-a- 
vis business partners and employees and attempt to measure integration and the willing- 
ness of those concerned to conform by linking a multitude of parameters and to generate 
corresponding pressure. However, as such systems usually operate in a non-transparent 
manner, they spur users on to even higher performance in accordance with the principle 
of intermittent reward. Examples of their use in the private sector include frequent flyer 
programmes, supplier relations in the automotive industry, variable pricing in online 
shops, or customer loyalty systems such as Payback or DeutschlandCard in Germany. 
Social scoring is especially questionable and violates dignity when it is used by the state 
as a sovereign instrument and is intended to induce fear-driven good behaviour among 
citizens by means of incentives or sanctions. 


2.2.8 Internet of Things 


2.2.8.1 Security Implications 


The fact that more and more “things” are now suddenly connected to the Internet 
and can be remotely read or even controlled is also putting previously ignored prod- 
ucts and manufacturers in the digital spotlight: Many manufacturers are suddenly IT 
companies without realising it. Their expertise may be in mechanical engineering, 
automotive production, production process control, or other areas. These companies 
have no experience in software development and no history of it. They usually have 
no technically qualified staff in the IT area, and in view of the shortage of skilled 
workers in this area, they will only get them at high cost. The lack of a corresponding 
software-compatible company culture is another problem. Networking and digitalisa- 
tion have given these companies additional status as software producers - without, 
however, being prepared for this task. In addition they are unable to create specific 
additional resources in the short term to cope with these new challenges and with the 
competition in their respective sectors. The result is usually that such new products 
from the Internet of Things have a pathetic security record, right up to renowned 
corporations such as BMW which have already had to deal with several hacks of their 
car models (BBC, n.d.). 
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2.2.8.2 Data Protection Implications 


In addition to extremely serious security issues, the connection of previously offline 
devices to the Internet also threatens the privacy of users. Basically, all usage patterns 
can now leave the device and be uploaded to the Internet, where they can be further 
processed in a variety of ways. Depending on the jurisdiction and the manufacturer’s 
compliance with the applicable data protection rules, linking such data with data from 
other areas of the user's life can give the evaluating institutions (companies, govern- 
mental agencies) information about lifestyle, political position, personal or financial 
circumstances, creditworthiness, and much more. And even if the linkage created by 
the manufacturer does not tally with the real world, the users still have to live with the 
prejudice of the machine, i.e. with a Digital Persona that bears the attributed character- 
istics in contrast to the real person. 
Interesting examples of usage patterns include the following: 


e Use of private coffee machines: Is the user potentially addicted to caffeine? How 
long does she work on weekdays and weekends? Does the user tend to drink 
coffee at certain times of the day? Is the user currently traveling? Does the user 
have visitors at the moment? 

e Data from eBook readers: What is the reading speed of the reader? What conclu- 
sions can be drawn about the user’s IQ based on the titles read and the reading 
speed? Is the user reading the sex scene of the book for the third time? 

e Smartphone charging history: Does the user keep the charge level always above 
70%, or does she regularly drain the battery? What conclusions can be drawn 
about the user’s creditworthiness (King and Hope, 2016)? 


As already mentioned earlier in the text, the legal process, ie. the application of the 
principles of the rule of law, is excluded. 


2.2.9 Transformation of the World of Work 


Not only has digitalisation created an enormous demand for IT staff - digitalisation is 
also beginning to transform certain professions or make them completely superfluous. 
It is not always only supposedly monotonous or simple activities that fall victim to 
automation, but also specialised activities that can either be significantly rationalised 
or even eliminated altogether by systems with AI components. (cf. Chapters 14 and 15). 


2.2.10 Lemon Market IT Security 


With the penetration of the real world by digital mechanisms, it must not be forgotten 
that not only pure functionality is replaced, but also the trust of users in the correct 
functioning of the goods or services they use. Especially in the product aspect of IT 
security, digitalisation initially brings with it a great many problems. Customers expect 
the digitalisation of their business processes to at least not lower their security level 


41 


I Humans and Digital Technology 


as a result ofthis change. However, as shown above, many companies are not original 
software producers and therefore have not learned or understood even the basics of 
secure software engineering. 

From the customer’s point of view, security is a feature that is difficult to understand 
because it is non-functional. It is not explicitly demanded by customers, but rather 
ignored or tacitly assumed. In any case, it cannot be practically assessed by this target 
group, because it would require both considerable IT expertise and insight into the 
internals of the specific products. In the case of the IT security market, we can therefore 
speak of a so-called lemon market, i.e., a market in which customers are unable to judge 
the primary features of the products offered for sale and are therefore dependent on 
clues provided by secondary features, or, in the worst case, on marketing statements or 
popularity scores (supposed customer ratings) of the manufacturing companies. Even 
IT experts usually fail in the assessment, as the internals of products are almost never 
voluntarily disclosed by the manufacturer with reference to intellectual property and 
company secrets. However, as customers cannot judge the safety of a product anyway 
due to the lemon market property, security is often neglected or omitted to save costs 
or even increase performance. At the same time, such a lack of transparency also feeds 
the category of so-called snake-oil products, i.e. products that intentionally carry no 
benefit, but instead intend to extract money from the pockets ofan uninformed clientele 
exclusively through flowery advertising statements. 


2.2.11 Energy Consumption of Digital Products — Example Blockchain 


Due to advancing digitalisation, the energy consumption of digital products and infra- 
structure is also steadily increasing — both in absolute and relative terms, in spite that 
energy efficiency is also improving. One information technology method that seemed to 
have great potential in the recent past is blockchain (or distributed ledger) technology. 
It has also only become widely implementable due to advancing digitalisation. The 
goal is to avoid a central, reliable, and trustworthy server instance that stores certain 
facts and selectively offers them for access - an example of this would be any kind of 
account management. The blockchain lacks such a central instance, which is why the 
term distributed ledger was coined for the basic application model. All information is 
signed and chained in multiple copies at participants of the system to make forgeries 
nearly impossible. Deviating information can be easily recognised by the other system 
participants. By identifying the participants only by means of crypto keys, the pseud- 
onymity of the transaction partners is guaranteed. It is important to note that only new 
entries are appended to the blockchain, but existing entries can never be changed. 
The size of the Bitcoin cryptocurrency blockchain based on this principle was about 
15 gigabytes (three feature films in HD resolution) in mid-2020. At that time, its size 
grew by about 1 megabyte per hour. However, Bitcoin is a fringe phenomenon in terms 
of its transaction volume - if the Visa credit card system were to be converted to block- 
chain technology, the growth of the blockchain would be assumed to be 1 gigabyte per 
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second, thus exceeding the dimensions of today’s IT technology within a day. Attempts 
have been made to reduce this growth through marginal losses in security, but the 
fundamental complexity could only be marginally influenced. 

Moreover, the energy footprint of the blockchain is devastating. To compare: While 
741 kWh were needed for a Bitcoin transaction in 2020, the same amount of energy 
would not have been needed to carry out even 100,000 credit card transactions on the 
VISA network - these would only have consumed 149 kWh of energy (Statista 2020). 


2.3 Artificial Intelligence 


Basically, the term “Artificial Intelligence” has been in use for 70 years and describes 
a melting pot of different techniques under this name. Therefore, it is comparatively 
unspecific. The following sections develop the term and subsequently narrow it down 
to machine learning and further to neural networks (see also Figure 2.10 ) - and thus 
to its current meaning. 


Neural 
Networks 


Machine 
Learning 


Artificial 
Intelligence 


Fig. 2.10: Relationship of the terms Artificial Intelligence (Al), Machine Learning (ML), and Neural Net- 
works (NN): NN is a true subset of ML and ML is a true subset of Al. (Source: The authors) 


2.3.1 General Term: Artificial Intelligence (Al) 


The term “Artificial Intelligence” is one of the typical buzz words of the digital age that 
has separated itself from the actual meaning of the word. It is driven by an intuitive 
understanding of the term “intelligence”, which could roughly address 


e multilevel, nontrivial insight, true to the etymological root intellegere (Latin, 
among other things “to show insight”), 
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e non-trivial, multi-stage reasoning involving hidden criteria and using formal-log- 
ical, but also heuristic methods, 

e a surprising, initially mysterious planning move (analogous to the Queen’s sac- 
rifice in chess) 


The latter was attributed to great explorers or generals such as Roald Amundsen or 
Napoléon Bonaparte. 

The transition to artificial intelligence (AI) is already linked to the early days of elec- 
tronic data processing, i.e. the “tube age’, when researchers tended to exuberantly forecast 
the emergence of an electronic super being, the all-knowing, superior “electronic brain”. 
In this enthusiasm, one of the founding fathers of computer science, Alan Turing, con- 
ceived the Turing test to determine the intelligence of a machine (Turing, 1950). In this 
test, machine intelligence is determined by a human interacting with hidden dialogue 
partners, some humans, some machines. If, after an appropriate time, the human cannot 
say with certainty which dialogue partner is a machine and which is a human, the ma- 
chines (rather, the programmes driving the machines) have proven to display at least a 
basic measure of intelligence. At that time, such “chatbots” were still science fiction. Now- 
adays they are already reality, even if their abilities are not at the highest intellectual level. 

With his ELIZA dialogue system (with an ironic twist) Joseph Weizenbaum demon- 
strated such a programme as early as 1966, which could pass the Turing test on a prim- 
itive level (Weizenbaum, 1966). 

Since its conception, the use of the term “Artificial Intelligence” has undergone sev- 
eral fundamental modifications. Clearly, intelligence in the human, neurophysiological 
or cognitive sense is not to be expected, so the following quotations, albeit with a slightly 
cynical air, contribute to a better understanding of the term: 


e “AI is whatever hasn’t been done yet” (Larry Tesler according to Hofstadter 
(1999)). 

e “Artificial intelligence has the same relation to intelligence as artificial flowers 
have to flowers. From the distance they may appear much alike, but when closely 
examined they are quite different. I don’t think we can learn much about one by 
studying the other. AI offers no magic technology to solve our problem. Heuristic 
techniques do not yield systems that one can trust” (Parnas, 1985). 


Thus, artificial intelligence in its conceptual vagueness was always a catch-all for the 
fringe areas of computer science still unexplored at any given time. Many methods that 
were still mysteriously attributed to AI 30 years ago have since passed into the canon 
of computer science or mathematics - for example, objects (originally frames (Minsky, 
1974)), genetic algorithms, linear and non-linear optimisation, or even statistical meth- 
ods such as cluster analysis. 

Using AI, intelligence is simulated, and the underlying methods can be completely 
divergent having no connection to intelligence as understood by humans. In the history 
of computer science, artificial intelligence has appeared, vanished, and re-appeared 
again and again in waves. Between the hype waves were the so-called AI winters, i.e. 
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phases in which AI was almost completely disregarded. With each re-appearance, how- 
ever, AI stimulated fantasies of superintelligence and, consequently, the end of humanity. 

In the hype of the 1980s, the so-called “logic-based knowledge representation” was 
predominant: rule-based systems were based on the modus ponens as a method of in- 
ference: I know that the rooster crows (B) when the sun rises (A). So, if I see the sun 
rising, I can predict that the rooster is about to crow (AB). 

As an “execution model’, rule-based systems dominated the market in the form of 
programming languages, such as Prolog with “executable” logic as its programming 
paradigm or as so-called “shells” with a motley assembly of logic-based methods. As 
end products, “expert systems” were offered that were supposed to perform in a manner 
comparable to human experts in a narrowly limited range of situations. They were also 
capable of justifying and explaining their assessments and decisions - this is very re- 
markable from today’s perspective, since such an explanation capability is hardly found 
in currently prevailing AI systems (see Section 2.3.3). 


2.3.2 Machine Learning (ML) 


The AI bubble of the 1980s burst because it quickly became apparent that knowledge in 
factual or procedural form is subject to constant change over time, and that it is impossi- 
ble to guarantee new knowledge is consistent with existing knowledge. The maintenance 
of an expert system, i.e. its regular adaptation to new conditions and requirements, very 
quickly turned out to be costly. A true simulation of a human expert also has to include 
a simulation of experience and further training, i.e. learning in the sense of adapting to 
changes in the environment, which could not be done by the AI of the 1980s. 

By the sound of it, machine learning would provide a remedy to this deficiency, 
but of course this is only simulated learning. This kind of learning is merely a kind of 
indirect programming: 


e Supervised learning: A system is presented with sample cases and sample solu- 
tions to “learn”; the “programmer” retains control over what a system is learning; 
the system is also learns the biases of the "programmer”- what it makes of them 
is initially unknown. 

e Unsupervised learning: A system is presented with examples of cases; the ma- 
chine develops its own conceptual system, and the people who operate the sys- 
tem then decide to what extent this might be useful. In case of doubt (and that 
will happen in many cases), they will discard unusable systems (trial-and-error). 


Inductive methods have emerged based on symbols and logic which, however, often lead 
to the exponential growth ofa rule base, since the consistency of the rule base is constantly 
in danger. At the same time, the ability of such a system to generalise tends to weaken. 
Mathematical-statistical methods are successful in finding regularities within mass data 
through regression or clustering. Therefore, they can simulate learning when executed 
on different or updated data sets. 
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The metaphor ofthe electron brain, as already emerged in the 1950s and constantly 
present in the background, led to the transition from the world of symbols to so-called 
“sub-symbolic” information processing. 


2.3.3 Neural Networks 


The metaphor underlying the term neural network refers to the neurophysiological ele- 
ments and structures ofthe cerebral cortex and the hope that, instead ofa few powerful 
processors as used in conventional information technology, a much higher number 
of primitive and relatively slow information processing units (“cells”) could achieve a 
crude simulation of higher-order cognitive processes. These complexes of primitively 
simulated “brain cells” influence each other by weighted transmission of their internal 
state (see Figure 2.11 (Kratzer, 1994)). 

External input is mapped into certain areas of such a structured network (“coded”), 
and the reactions of the network are tapped from other areas (“decoded”). Effective co- 
ding and decoding are very important for good overall performance of such a structure. 
“Learning” is essentially done by adjustment of the weights describing the influence of 
cells upon each other. Using methods of numerical mathematics, “learning” boils down 
to a search for extreme values in high-dimensional, nonlinear systems of equations. 

Thus, a neural network is kind of a weight tensor, i.e. a highly complex constellation 
of weights which, in conjunction with the coding and decoding procedures, contain 
the “knowledge” of a network. The knowledge is represented in a way distant from 
human reasoning and language. The creator of such a network does not explicitly write 
down the weights; rather he/she does teaching sessions with case samples and sam- 
ple solutions (“supervised learning”) or allows the structure to watch its surroundings 
(“unsupervised learning”). We should always keep in mind that the use of the term 
“learning” in this context is rather pretentious - nothing more than a numerical extreme 
value search in complex, nonlinear systems of equations, far away from intelligence as 
defined by humans. 

For example, the (admittedly simple) configuration shown in Figure 2.11 could be 
used to classify letters in a 6x6 black and white raster: For each connection between 
the cells, a weight is stored as a measure of the influence of one cell upon another. The 
black-and-white raster is mapped onto a so-called input layer with 36 cells. This, in turn, 
influences the eight cells in the intermediate layer, and these, again, influence the eight 
cells at the bottom where the classification results can be retrieved. The “knowledge” 
stored in a neural network depends on the: 


e topology of the network (number and connection of processing units): How 
many input/output layers and how many intermediate layers are there? How 
many neurons are assigned to each? 

e initialisation of the structure: What are the initial values for the weighted con- 
nections in the network? 

e number, sequence, and quality of training samples; 
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Fig. 2.11: Example of a simple neural network recognising letters based on pixels in an image. (Source: 
The authors) 


e setting of parameters, for example: How will deviations from the ideal behaviour 
affect the weights between neurons? Abrupt weight changes may cause oscilla- 
tions in the structure that cannot be controlled; 

e method of coding input and decoding output. 


From an external perspective, a neural network is a black box that, in tests and field 
trials, is either performing satisfactorily or unsatisfactorily. In most cases, a full-scale 
validation is not possible due to the underlying problem complexity. Many of the net- 
works created by training are simply discarded because they do not fulfil expectations. 
The proponents of neural networks claim the inevitability of arriving at a solution by 
“training” Such a claim, for all practical purposes, is highly unrealistic. Since as a result 
of the high dimensionality of the weight tensor, a test is only possible selectively, large 
parts of the behaviour of a trained network remain in the dark. Thus, the reliability 
of such a network in practical use is narrowly limited, since sudden and inexplicable 
deviations from the intended behaviour might happen at any moment. 
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When designing and training a neural network, two dualisms usually emerge, for 
which a workable compromise has to be found by experimentation: 


e Reproduction vs. Generalisation: Do I want my training samples to be reproduced 
cleanly and without deviations, with possible limitations in interpolative perfor- 
mance? Or is the focus on interpolative performance, with a possibly less than 
perfect reproduction of the training patterns and thus a possible reduction in 
reliability? These contradictory requirements are reflected in different network 
topologies, different parameter settings (depending on the network model), as 
well as different training strategies. The following example illustrates the prob- 
lem. Let us assume that the following identification numbers are classified as a 
certain category: 


462 294 193 306 986 
348 202 206 776 872 


A reproducing network would react to exactly these identifiers and ignore any 
visible regularities, while a generalising network would recognise that these iden- 
tifiers are mostly even numbers and ignore the exceptional case 193 as “noise”. 
This unusual number might however be significant. From an application point of 
view, reproducing networks are not worthwhile because there would be cheaper 
ways of reproduction, but if the focus is on generalisation (ultimately a kind of 
interpolation), there is a danger that crucial facts will be “ironed out” Ultimately, 
this leads to the assessment of how many false-positive or false-negative classifi- 
cations are acceptable. 

e Stability vs. Plasticity: To what extent does the network follow new external in- 
fluences that may contradict previously trained patterns, with the danger of an 
“oscillation” of the network behaviour? How stably does it behave in the face of 
such contradictory input - this is particularly important for neural systems that 
adapt continuously without human interaction. 


No matter what decisions are made when creating a neural system, the actual perfor- 
mance is reduced to pure input/output behaviour. The “learning”, or rather the adapta- 
tion, is based on the subsymbolic approach, which then leads to a neural structure de- 
veloping its own terminology, which is not comprehensible to a human monitor. From 
this follows that there is no comprehensible explanation or justification of individual 
evaluations or decisions. Neural systems require blind trust. Depending on the opera- 
tional scenario, this can have potentially significant, if not devastating, consequences 
for those affected. 


2.3.3.1 Deep Learning 


Deep Learning (Bengio, LeCun, and Hinton, 2015) sounds mysterious as a term at first, 
but it is not a conceptually novel approach. In principle, it addresses neural networks 
in which the number of neuron layers between input and output has been massively 
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increased. This is not conceptual progress, but it allows training of certain parts of the 
network in advance. It is no coincidence that the indisputably great successes of such 
networks are in the field of image processing, in which so-called features, i.e. charac- 
teristic sub-areas, in combination lead to the classification of an overall image, such as 
“There is a kitten in the picture”. The fact that neural networks and “Deep Learning” 
are currently being propagated as tried and tested methods for artificial intelligence 
cannot be attributed to an actual paradigm shift, but only to the immensely increased 
computing power available today compared to the 1980s, the much wider availability 
of data, even from the most intimate areas of life, as well as the willingness of broad 
sections of the population to share, assess, and react to digital stimuli. 


2.3.4 Big Data 


The term Big Data combines a wealth of different methods from computer science, but 
also from applied statistics, aiming at 


e analysing very large amounts of data from different sources and revealing previ- 
ously unknown regularities and correlations, 

e deriving from such regularities predictions of the future behaviour of the society 
or of individuals, but also 

e classifying individuals and phenomena. 


Such visions have been developed very early - just think of Isaac Asimov’s vision of a 
fictitious science, which he called psychohistory, set up to allow forecasts of social, eco- 
nomic, even military events and processes based on data from the past (Asimov, 1991). 
We are now, some 60 years after the advent of mass data processing, able to carry out 
such analyses in almost all areas of life, based on many criteria and a speed of processing 
much higher than would have been possible using human analytical work. This includes 
not only a quantitative dimension but also a qualitative dimension. Digital machines 
sometimes correlate data in ways that, at least at this point, no human has come up with, 
and which appear to better reflect reality. Of course, machine learning, especially neu- 
ral information processing, has lately been increasingly used for this purpose, with its 
known pitfalls and especially the lack of justification for decisions. Applications are, for 
example, the “suggestion lists” familiar from Internet commerce or streaming services, 
but also credit ratings and classification of applications in recruitment procedures - 
right up to official procedures (e.g., entry into the Schengen area for immigrants without 
EU citizenship (Gallagher and Jona, 2019)). 

It is important to note that information from many data sources, including contact 
and movement data, medical data from fitness bracelets, and - in case of physical con- 
tact - criteria such as eye movements, muscle twitches in facial expressions, or even the 
charging status of a mobile phone may be relevant. 
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2.3.5 Aland Us 


The use of AI technology within application programmes targeting our personal sphere 
of life harbours a still drastically underestimated danger to our freedom and our inter- 
ests, e.g., when touching on our health, our financial affairs, or our freedom. The dan- 
ger is that we subject ourselves semi-consciously or unconsciously to an interest-driven, 
obscure catalogue of criteria to which we submit voluntarily or by law. This is especially 
true in the case of neural systems, since they refuse to explain a judgment or decision. 
Of course, free people cannot be prevented from submitting to such a regime by their 
own volition except when this is necessary for their own protection, or for the protec- 
tion of the state and the legal system. Nevertheless, even in Germany there are areas, 
in which people have no real choice, for example, for a credit rating. In such cases, 
there must as a matter of principle always be an option for a complaint to a court. This 
has also been recognised by data protection agencies: Here, the first demand usually 
is for a disclosure of the methods used. However, this leads to the question as to what 
should be disclosed. The method itself, i.e. the algorithm, is usually known from the 
scientific literature and is therefore openly available - at least in its most basic form. The 
next step would be to reveal the underlying network structure and coding/decoding 
algorithms. However, this is of little value if the data used to “train” the network are 
not disclosed at the same time. Such training data are either deemed to be company 
secrets, or it is claimed that revealing the data would violate privacy rights of persons. 
Even if all that has been revealed, the soundness of an AI system is hard to judge even 
for experts - if at all. 

Another approach to the absolutely necessary regulation of this new technology 
would be a requirement to explain and justify every single decision. However, this would 
practically exclude the use of neural technology, since such explanations and justifica- 
tions are simply not available. 

Proponents of AI technology often claim that such systems are neutral and objective. 
However, this depends on the data available for training, and it is quite possible to 
discriminate without using explicit criteria such as gender, age, or skin colour. 

From text analysis, for example, it is statistically known that 


e women use the word T more often, 

e men, on the other hand, definite articles, 

e younger people more often use auxiliary verbs and 
e older people use prepositions more often, 


which opens the door to indirect discrimination, simply by analysis of a written text 
(Newman et al., 2008). 

Neural systems are known to have developed sexism and racism simply by analysing 
social interactions. Artificial intelligence is thus not able to neutralise prejudices deeply 
rooted in society, or to observe universal values - instead, it represents a mirror of 
the recorded, possibly biased input data (UN News, 2020). However, there might also 
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be deliberate manipulation of such systems: a well-known case was the chatbot Tay 
launched by Microsoft (Hunt, 2016), which was deliberately “re-educated” to racism 
by malicious users. 

What is the relationship between such intelligent systems and their operators? It is 
often emphasised that these systems are “only supportive” and that “the human operator 
retains full control and the final decision”. This goodwill attitude can be fairly assumed 
for the majority of engineers working in the field of AI. Looking closer, however, we 
quickly find out where this intellectual backdoor falls short: it is already known from the 
early days of decision-support systems that this “support” can rapidly give rise to pres- 
sure to justify why a machine assessment or recommendation was overridden, which 
would de facto turn the mere proposal of an assessment and recommendation into an 
order (Langer, König, and Busch, 2020, cf. also the following third chapter). 

It is important to always keep in mind that any parallels between human intelligence 
and neural artificial intelligence are purely metaphorical. The determining factor in hu- 
man action is not only plain rationality but also emotion, common values, often individ- 
ual or socially shaped moral concepts, as well as the preservation of one’s own existence 
or that of the group or species. A neural network, on the other hand, is a machine and as 
such implements a highly complex interpolation algorithm. The previously mentioned 
universal or personal values may or may not be reflected in the training examples - what 
then happens in non-trained situations, however, remains open. Artificial intelligence 
is a tool without life, self-reflection, bodily experience, the concept of lifetime and thus, 
without concern for one’s own existence. 

Commercial artificial intelligence systems predominantly deal with application 
fields whose complexity cannot be grasped by conventional programmes. Therefore, 
they cannot be tested and validated following the rules of conventional software devel- 
opment. In case of failures, the manufacturers usually promise improved performance 
in the indefinite future - but nothing more. This is not serious if the application field is 
harmless and no or only minor damage can occur (for example, in games or for harmless 
forms of marketing, such as suggestion lists). In critical fields of application such as au- 
tonomous driving, legal processes or complex surgical procedures, however, the already 
existing and tested approval and licencing procedures with the usual documentation 
and validation rules should ideally be applied. However, it must be clear this will make 
the use of machine learning considerably more difficult, if not impossible. 

Finally, it should be mentioned that there are already substantial efforts in this area 
both nationally and internationally. These attempt to stem and limit the unregulated 
flood of classifications and analyses with negative consequences for those affected, and 
the marginalisation of people that is particularly evident in the labour market. Examples 
include: 


> 


e the European Union’ “Ethics Guidelines for Trustworthy AT”, 
e the publications of the Data Ethics Commission of the Federal Government, 
e the Ethics Guidelines of the German Informatics Society (GI). 
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All these reflect responsibility for our societies and the willingness to regulate AI to 
benefit all stakeholders. To what extent such principles will survive the confrontation 
with economic interests remains to be seen. 


2.4 Summary and Outlook 


In contrast to the widespread technocratic utopias, or dystopias, of a digitalised eco- 
nomic and social structure, the well-founded discussion of the underlying technology 
is rather sobering. However, the sometimes naive or techno-optimistic overreliance on 
digitalisation and AI in the economy, society, politics, and the private sphere is based on 
such positive assumptions which, in most cases, are untenable on closer examination. 
In summary, the following points should be emphasised: 


e The capacity of the underlying computing technology is limited. Capacity limits 
are constantly being expanded, but this expansion is struggling to keep pace with 
the demands of increasingly complex tasks. 

e Nevertheless, it is possible to process and condense information on individuals 
and thus to characterise, advise, and in the worst case, scrutinise individuals to 
an ever-increasing degree. 

e Digitalisation - similar to economisation since the 1980s - is increasingly pene- 
trating weakly structured areas of the economy, society, politics, and the private 
sphere, which leads to decision-making spaces with a multitude of degrees of 
freedom. The underlying technology cannot keep pace with the complexity of 
human decision-making, but nonetheless might dominate it. 

e Currently, so-called artificial intelligence is primarily driven by increased com- 
puter performance and only to a small extent by conceptual progress. We are 
still a long way from a superintelligence in the sense of the trans-/posthumanism 
presented in chapter five. 

e The use of digitalisation is possible, regardless of the area of application. However, 
human individuals will perceive the behaviour of a digital system as increasingly 
erratic, surprising, or even threatening in application fields of high importance. 

e The cohesion of our society is shaped by discourse: at an appropriate level, dis- 
course is probably possible with the developers of an application system, but not 
with “intelligent” application systems themselves. 


The penetration of digitalisation into a large part of our professional and private spheres 
has already taken place - a “rollback” to the analogue world would be unrealistic. An 
open democracy, however, needs a discussion of values that places both freedom and 
the dignity of every human individual at the centre of this discourse and which, in its 
implementation, leads to comprehensive regulation at all levels. This is exactly why we 
have written this book! 
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Path Dependence and Lock-in 
Jasmin S.A. Link 


In the public debate, many suggest we are currently in the midst of a fourth industrial 
revolution. A main part of this revolution is the digital transformation of society. In 
order to stay internationally competitive, there appears to be no other way than a sub- 
stantial digital transformation in Europe. The industrial sector is already digitised to a 
great extent. Politics and education would need to speed up to keep track. Therefore, 
schools should utilise the granted digitalisation funds even without a pedagogical con- 
cept to achieve a quicker digitalisation of the education system. But why? And where 
does this development lead? 

Path dependence theory can be used as analytical basis to understand the speed of 
the development and the perceived inevitability of follow-up decisions. Path-dependent 
processes show exponential dynamics, historical manifestations and interdependencies, 
as well as cognitive and economical dependencies. There are, for instance, marketing 
agencies that try to use path dependence to retain customers through mutually de- 
pendent product packages. Others try to suggest one would already be in a locked-in 
situation and there would only be the choice to comply (but not to deny). Generally 
spoken, path-dependent processes generate dependencies and induce social dynam- 
ics. Furthermore, the analysis of path-dependent processes can help to identify future 
developmental dynamics reliably. In case of unintended direct or indirect effects of 
path dependence that cannot be anticipated, deliberate measures to counter such effects 
should be taken as early as possible. 

Various definitions of path dependence can be important when focusing on digi- 
talisation, IT-networks, and artificial intelligence (AI). Notations may vary depending 
on the object of concern that is analysed or discussed, the background of the scientist, 
and potentially also the observational level or the dynamics the object is embedded in. 

When analysing path dependence in the context of digitalisation, IT-networks, and 
AI, the following diverse perspectives are used: 

Digitalisation is coined by routines and supported by institutionalisation and stan- 
dards. Some procedures do change - but in a coherent way - towards increasing digital- 
isation and digital analytics. Reactive sequences of historical events can have profound 
impacts, similarly organisational fields, individual experts®, or social networks. Learn- 
ing effects pay off and procedures are optimised incrementally. 


9 Personal or individual experts do not necessarily count objectively as experts. However, these people 
function as experts for the individual decision-maker. In this sense, also a best friend that always has a 
clue or a mentor could be used as expert, even if that person does not have specific expertise to answer 
the given question. 
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In this context, networks are very important. In interconnected procedures, the 
digitalisation of one process necessitates the digitalisation of another one. Therefore, a 
well-functioning network needs a certain standard underlying equipment. Hardware 
and software need to be compatible, too. Then the network allows for a smooth and 
quick data exchange, data analysis, and data use, potentially even to a greater extent than 
is feasible in an analogue manner. 

So-called AI can be applied in data preparation and data analysis. Increasing digi- 
talisation and increasing interconnectedness are raising the availability of linked data, 
so that IT analysis tools are necessary to actually utilise these big data. Prior to deci- 
sion-making, these big data need to be visualised, analysed by predefined criteria, and 
controlled for missing data or presumably erroneous data. Different hypotheses on these 
big data require different methods for testing. However, there is the substantial caveat 
that no one can personally recalculate or even check the iterative steps of a neuronal 
network. With increasing availability of electronic data, growing interconnectedness, 
and reinforced digitalisation, more and more human decision-making processes will 
depend on the results of the applied AI systems. 


3.1 Definitions of Path Dependence and Lock-in 


The general understanding of the term “path” does not necessarily coincide with the 
definition of path dependence and not every definition of path dependence includes 
the terminal state of a lock-in. After explaining the different perspectives of path de- 
pendence and several possible definitions of a path, the remainder of this chapter will 
use a process-oriented definition of path dependence: A path-dependent process is a 
self-reinforcing process with a tendency to reach the state of a lock-in. A process has 
reached the lock-in (i.e. it is locked in) if changes of the given state do not occur any 
longer or occur only incrementally. 


3.1.1 Digression to Mathematics 


Many scientific disciplines use mathematical definitions. Regarding path dependence, 
different understandings can be generated from different branches of mathematics. 
Therefore, the following small digression outlines these understandings in this other- 
wise sociological chapter. 

For example, in graph theory in network analysis the notation “path” describes a 
chain of multiple edges in a given network (cf. Figure 2.1 in chapter 2): A path is a 
sequence of edges that connects nodes, which might not share an edge. The number of 
edges in this sequence is the length of the path"®. Networks can be used to analyse many 
structures or bottlenecks, e.g. of a power grid, a logistic network for industrial products, 


10 E.g., the longest path in the connected graph in the middle of Figure 2.1 has the length four. It connects 
two nodes that are not directly connected by an edge. 
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or a consumption network consisting of numerous households. Social networks can be 
analysed as well, e.g. in epidemic contact analyses or when analysing the diffusion of 
information. The topics of possible applications are diverse. However, this notation of 
a path is merely static and therefore, there is no understanding of an associated lock-in. 

In a different subfield of mathematics, in stochastics, a path is also defined as a 
sequence of edges in a network, in which the nodes are events that occur with a certain 
probability. The structure of such networks is a tree, which means from a first event a 
certain number of edges branch as there are alternative subsequent events, from which 
again branches lead to further events. In this tree of connected events, a path denotes 
one potential sequence of events. Knowing the probability of each single event along the 
path allows for calculating the expected value of the final event of a path. If there is more 
than one path leading to the same final event, the expected value can vary depending 
on the path. The total expected value of a certain event depending on a specific starting 
event is the sum of all expected values of that particular event over all paths leading from 
the starting event to the given final event. Such expected values are for instance used in 
decision theory in business economics. Again, this understanding of path dependence 
does not show any dynamics. Therefore, no understanding of a manifestation such as a 
lock-in is included in the expected value theory. 

Instead, if you consider a dynamic system, there is potentially no longer a countable 
set of alternatives. It is possible that different dynamics can be described, depending on 
the initial value. Some of these dynamics may have an attractor, which they approach 
and from which the dynamic process does not depart anymore after it has been reached. 
The plot of such dynamic can be interpreted as a path towards the attractor, the lock-in. 
If the process is accelerating the closer it gets to the attractor or is increasingly spatially 
limited close to the attractor, the process can be characterised as a self-reinforcing pro- 
cess with the tendency towards a lock-in. A physical example for such a process can be 
a pendulum that swings around a magnet, by which it is more and more attracted the 
closer it gets to that magnet. 

In Sociology, these perspectives described above can be combined. The basic defi- 
nition of path dependence includes the process perspective of a self-reinforcing process 
with the tendency towards a lock-in. Such processes are practically self-reinforcing dy- 
namic processes on a weighted directed graph, which approach an attractor. 

In sociological terminology, this matches the self-reinforcing processes with the ten- 
dency towards a lock-in, whose diffusive effects can be analysed in social networks, or 
weighted directed social networks, in which the weight of the edge correlates with the 
probability of a specific decision to act. 


3.1.2 Path Dependence Explained by the Example of the 
QWERTY-Keyboard 


As in Mathematics, sociologists use different notations of “paths” matching different 
subfields of Sociology or the perspective that is taken based on the neighbouring dis- 
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cipline as well. That way, examples are often analysed and discussed from multiple 
perspectives. Two classics in the sociological path dependence debate are the analysis 
of the development and diffusion of the QWERTY-keyboard (David, 1985) and the 
applicability of a Pólya urn model, in which the balls drawn are replaced and another 
ball of the colour just drawn is added as well (Arthur, 1994). 

The historian Paul A. David assessed the development of the QWERTY-keyboard his- 
torically (David, 1985, 1997, 2001, 2007). He recognised that even on the early keyboards 
the placement of keys is the same as on the keyboards of later computers. But why? 

David discovered that in the development of typewriters the typewriters were op- 
timised in terms of writing processes, especially regarding the writing speed. When 
two bars were hit right one after the other, they had a higher probability of clashing or 
jamming if they were adjacent to each other - in contrast to bars further apart. Clashing 
bars always interfered with the writing dynamics because one needed to stop completely 
to release the bars before continuing to write. Thus, it was important to minimise the 
likelihood of bars clashing or jamming in order to optimise the overall writing speed. 
Accordingly, the English language has been analysed to calculate how often particular 
words occur and especially how often specific letter combinations occur in these words. 
Afterwards, the keys were placed on the bars to make it most unlikely that letters, which 
are often found adjacent in words, are placed next to each other on the typewriter as 
well. Additionally, one could remark that all letters of the word “TYPEWRITER” have 
been placed in the topmost row of the English keyboard. According to written records, 
this goes back to helping early salesmen to find the letters of their product name more 
quickly, even though they might not have been trained typists at that time. 

But why did the placement of keys remain the same on the electronic typewriters, 
the computer keyboards, or even modern tablets or smartphones? There are no bars 
anymore that could clash or jam, nor does the brand name TYPEWRITER exist any 
longer. On tablets or smartphones there are not even keys anymore and it is likely that 
letters are not even typed with two hands, rather with one hand or even single fingers 
or even only one or two thumbs. Also, it can even be assumed that modern English has 
changed so much from the English at the time of the invention of the typewriter that 
even the likelihood of adjacent letters has changed in comparison to 100 years ago. 

This phenomenon is called path dependence. “History matters” describes the dis- 
tinctive influence of early decisions or events on later developments in a non-reversible 
chronology. And David (1985) has pointed out that “[...] while they [the agents] are, as 
we now say, perfectly ‘free to choose; their behaviour, nevertheless, is held fast in the 
grip of events long forgotten[...]”" 


11 Some economists deny the mere existence of path dependence since their theories do not allow for 
any inefficiency (cf. Liebowitz & Margolis, 2014). Other economists come up with arguments how 
decisions in the historic context of the development of keyboards can be considered to be locally 
efficient nonetheless. There are learning effects, effects on the workforce, marketing effects, effects of 
scale, the theory of sunk costs, or a market efficiency at a higher level. 


56 


3 Path Dependence and Lock-in 


It can be assumed that once a person has learned to type on a QWERTY-keyboard, 
finding the correct keys on that particular keyboard layout is more efficient compared 
to an alternative layout such as the DVORAK-keyboard”. Thus, once early secretaries 
learned to write on the new typewriters using QWERTY-keyboards, retraining them 
would cost money that an employer could save when offering them a typewriter with the 
QWERTY-keyboard in the office. Furthermore, if an employee needs to be replaced, the 
replacement is likely chosen to be able to type on the available typewriter as well. Over 
time, the increase in demand of secretaries with typing abilities led to the official edu- 
cation of future secretaries in typing on typewriters. Accordingly, even more employers 
had to buy typewriters as basic equipment for their offices. Public awareness increased 
as well and at some point offices could no longer be imagined without typewriters. Ex- 
changing an established typewriter by a model with a different keyboard layout would 
imply that previous investments in hardware and the education of the secretaries had 
to be accounted for as sunk costs. 

Even with regard to the typewriter production it is likely that maintaining the estab- 
lished keyboard layout led to increasing returns to scale and reduced unit prices. Similar 
reasons, particularly learning effects and labour market dynamics, can be discovered in 
the subsequent substitution of mechanic typewriters by electronic ones as well as in later 
product developments, in which the mere “writing” was no longer an essential feature. 
Presenting the product in an advanced but still comforting traditional way might have 
helped marketing efforts and increasing consumer acceptance (Beyer, 2005). 

The following examples highlight further perspectives to discover and analyse path 
dependence: 

Historical sociologists describe reactive sequences, which are causal chains of events 
in which early ones trigger and increase the effects of subsequent events (Mahoney, 
2000). Institutionalists stress that established institutions only change incrementally 
(North, 1990), which can be interpreted as an optimisation of the institution or as a 
lock-in effect. Political scientists apply the concept of path dependence to explain polit- 
ical dynamics (Pierson, 2000) and emphasise the unpredictability of the future path at 
the very beginning (Collier & Collier, 1991). However, innovation, management, and 
organisation researchers consider how any involved agents can learn to actively cre- 
ate paths, which implies initiating and shaping them (Garud & Karnge, 2001). Sydow, 
Schreyögg and Koch (2005, 2009) have designed a three-phase-model (Figure 3.1) to 
visualise contingency at the beginning of a path as well as the self-reinforcing formation 
of the path in the second phase until only incremental changes are still possible in the 


12 The so-called "DVORAK“-keyboard was designed to be particularly efficient with regard to ergonomic 
considerations. However, it could not overcome the market power of the established QWERTY-key- 
board due to path-dependent processes. Until today, it occupies just a niche of the keyboard market and 
is used to only a limited extent. On mobile devices, there is nowadays also the possibility of an alphabetic 
keyboard layout. However, if you have learned to type quickly on the QWERTY-keyboard, your fingers 
usually find the correct keys almost by themselves. In that case, an alphabetic keyboard layout would be 
counterproductive, as you would have to think actively while typing where the keys are actually placed. 
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Fig. 3.1: Three-phase-model of the development of an organisational path (Sydow et al. 2009). 


lock-in. However, taking a consulting perspective, the determination of a lock-in is 
dissatisfying because the three-phase-model does not consider any exit from the lock-in. 

Organisational sociologists combine multiple perspectives by describing path de- 
pendence as a process, which is self-reinforced by accompanying mechanisms and 
manifested to the point of lock-in. The assessment of the logic of continuity of each 
mechanism also allows for the identification of chances for destabilisation (Beyer 2010, 
Table 1, cf. also Table 3.1). 


3.1.3 Increasing Returns — Growth Dynamics That Can Result In a Lock-in 


At about the same time as Paul A. David, W. Brian Arthur (1989, 1994) developed his 
theory of increasing returns. His fundamental question was how one could predict 
which technology would win in a market? 

Even posing this question invokes economists’ critique because according to Adam 
Smith the invisible hand as a free market mechanism selects the best technology. Thus, 
if there are clear preferences, complete information, and universal efficiency, the best 
technology succeeds in this perfect market. Arthur’s question connotes that it is possible 
to think of markets, in which perhaps a suboptimal technology could win. That way, 
he contradicts the universal applicability of standard economic theory (Arthur, 2013). 

Arthur observed increasing returns, i.e. growth trends intensified. Unit costs and 
thus unit prices could be reduced by increasing returns to scale in production processes, 
which can lead to further increases in sales. He formulated the hypothesis that a technol- 
ogy can win in a market even though it is not the best technology by objective criteria 
if increasing returns are present (Arthur, 1994). 

Arthur uses a Pélya urn model to describe and demonstrate the positive feedback 
mechanisms (Arthur, 1994), a concept that is well known in stochastics. 
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Polya urn model: At the beginning, there is an urn with two balls, one blue, and 
one red ball. In each step, one ball is drawn from the urn. After noting the colour 
ofthe ball that was drawn, the ball is returned. Furthermore, an additional ball of 
the colour that was just drawn is placed in the urn. Consequently, the probability 
of drawing a ball with a particular colour changes after each step. At the beginning, 
the ratio between colours is 1:1, at the next step it is 1:2. Then it is possible that the 
colour ratio is 2:2 but it is more likely that it is 1:3. The probability that a ball of the 
majority colour is drawn increases more and more so that existing discrepancies 
are manifested. However, balls drawn at the beginning have a more pronounced 
influence on the final ratio of colours. If there already are numerous balls in the 
urn, an additional ball only has a marginal effect. At a certain level, the colour ratio 
in the urn tends to stabilise. However, it is impossible to predict this final ratio at 
the beginning of the experiment. 


3.1.4 Definition of Path Dependence and Lock-in in This Chapter 


In this chapter, the definition of path dependence is: A path-dependent process is a 
self-reinforcing process with the tendency towards a lock-in (Link, 2018, p. 3). 

This definition can be used as basis for the assessment of path dependence from 
various perspectives. A self-reinforcing process has the effect that early events play a 
more crucial role than later ones (history matters). The self-reinforcing process of the 
QWERTY-keyboard (David, 2001) can be determined in historical retrospective as hav- 
ing reached the lock-in very quickly, and manifesting mechanisms can be identified that 
stabilise the lock-in. “Increasing returns” (Arthur, 1994) are one of the mechanisms that 
can describe the self-reinforcing dynamics of a path-dependent process. Additionally, 
increasing returns can lead to a manifestation of a process towards the lock-in (cf. urn 
model, Arthur, 1994). Reactive sequences can describe the self-reinforcing dynamics by 
chains of subsequently triggered events and simultaneously highlight the non-reversal 
chronology by the tendency towards the lock-in. 

This definition does not include any specifics at the beginning of the process. But by 
defining the self-reinforcing dynamics of the path-dependent process, even tiny differ- 
ences in initial values can generate large impacts on the resulting lock-in very quickly 
(Arthur, 1994; Collier & Collier, 1991; Vergne & Durand, 2010). 

In the following, this definition will be used to combine the perspective of agency 
with a network perspective to apply it to the digitalisation, IT-networks, and AI. 


3.1.5 Path Dependence as Following Behaviour — Path-Dependent Networks 


In this part, the micro level of a path-dependent process is observed and the question is 
what happens to agents who are involved in a path-dependent process? Subsequently, 
these insights from the micro level can be aggregated back to a meso level to generate a 
network perspective based on the definition of path dependence. 
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In the historical analysis of the development and the manifestation of the 
QWERTY-keyboard usage, Paul A. David already noted that even though agents are 
completely free in their decisions, their decisions depend strongly on former events. 
Remarkably, in this process even events can play a role, which none of the decision-mak- 
ing agents can directly remember. How can that be? These events are not part of legal 
texts or written on posters pinned to the wall with the note to always act accordingly 
no matter what happens. Instead, path dependency shapes the involved agents’ actions 
so that they do not even think about alternatives. Therefore, they underestimate their 
own ability to act differently or are unaware of the necessity to leave the prescribed 
path-dependent process. 

There are different mechanisms that are described in the previous parts in more 
detail, which lead to self-reinforcing dynamics in the path-dependent process. How 
does the path-dependent process affect a particular and any future decision-making 
processes? This can be analysed as follows (Kominek, 2012; Link, 2018, pp. 32ff.): 

Looking more closely at the decision-making process of a single agent reveals that 
the agent has diverse alternatives to design the action decision-making process: for 
instance, spontaneously, value-rationally, purpose-rationally, or preference optimised 
according to an individual usability function. However, if this agent and the related 
decision are involved in a path-dependent process, this process will generate similar 
decision-making situations for the agent. Of course, the agent could again apply the 
same detailed decision-making process as before. But if several situations, in which 
decision-making is required, resemble each other, maybe the same action that has re- 
sulted from the first decision fits the second situation as well. Then simply applying the 
same action is quicker than starting an intensive decision-making process all over again. 

The “least-effort-principle” (social psychology) states that whenever the brain has 
multiple ways of pursuing a goal it takes the one that requires the least effort. Applied to 
the decision-making situation of the considered agent, facing comparable decisions the 
agent increasingly merely applies the actions from former decisions again. Thereby, the 
agent develops a routine. Accordingly, also the agent’s approach to new decision-making 
situations changes: Instead of acting value-rationally, purpose-rationally, or preference 
optimised like before, the decision-making process is shortened to a routine application. 

But what happens, if this imprinted agent faces a new situation? At first, he might try 
to apply the old routine again. Either the old routine still works or it is no longer appli- 
cable. In the latter case, the agent needs a new routine that matches the new situation. 
However, developing routines is time-consuming. Instead of developing new routines, 
it is quicker to adopt an action draft from an external source that matches the new 
situation. This could be another agent, an institution, or the masses. Due to the least-ef- 
fort-principle, the agent would choose to adopt a draft for his action. Consequently, even 
ifthe agent optimised his preferences in first place, he increasingly only applies available 
action drafts as his decision-making process is shaped by path dependence. The result is 
a following behaviour: Influenced by path dependence, the agent develops the tendency 
to follow other agents or the masses with his actions (Link 2018, p. 33). 
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Under the influence of path dependence, the tendency of one agent to follow other 
agents or the masses can turn into another routine for decision-making. Consequently, 
the agent can also apply this following behaviour in decisions to act, which are not di- 
rectly related to the path-dependent process (Kominek, 2012; Link, 2018, Chapter 4). 
Such following behaviour can become a routine again, which further reduces the prob- 
ability of even looking for alternative ways of decision-making. 

Focusing on one agent who shows following behaviour, an egocentric network can 
be created that maps the sources of the action drafts such as other networks or indicators 
of aggregated behaviour of the masses. This path-dependent social network (PDSN) 
(Kominek & Scheffran, 2012) can be understood as an egocentric social network, in 
which e.g. the agent consults his computer expert, rests his consumer decisions on Am- 
azon rankings, or uses the Google score as basis for spelling decisions. Likewise, at the 
airport, an agent A accidently recognises that a traveller B takes the same flight, follows 
B to get to the correct check-in and to the right departure terminal later-on. Traveller B 
does not necessarily need to recognise that person A is following him. At the same time, 
traveller B, who is followed by A, follows another agent C. Then it is possible to describe 
a PDSN for the second agent in the row, B, mapping his following behaviour, which 
agent C is part of. And by chance, agent A follows agent C indirectly, unknowingly, and 
without knowing C. 

This has numerous consequences, for example: 


e Via cascades of such coupled PDSNs, the German phase-out of nuclear power 
can be explained as a result of the effect of the Japanese earthquake that occurred 
close to Fukushima in 2011 (Kominek & Scheffran, 2012). 

e Knowing such PDSNs and the probability with which the egocentric agent per- 
forms a following behaviour and applies this network accordingly, the agent’s 
decision can be locally temporally approximated (Link, 2018, pp. 143ff.). 


3.2 Path Dependence in the Digital Transformation 


Based on the definition of path dependence as a path-dependent process, there are 
several levels available for the application to the digital transformation: a macro level 
perspective of dynamic processes, a micro level perspective of single agents who decide 
on actions, and a meso level perspective of interconnected PDSNs of agents, which 
essentially map following behaviour. 

If you consider digitalisation as a process that includes the introduction of hardware 
and software, many mechanisms can be identified that stabilise the development path 
and lead to the self-reinforcing characteristics of the process. For example, learning 
effects occur as soon as many employees have the skills to work with digital methods, 
data, and documents. Then more and more digital processes can be included in the 
overall workload. At the same time, working processes that include digital processes 
such as digital documentation are going to ask for employees with the relevant skills 
once the positions are vacant again. 
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Educating new employees is also possible, but of course incurs costs. Furthermore, 
whenever digitalisation leads to better informed decisions or accelerates the proceed- 
ings, increasing returns occur: The more decisions are made based on analyses of digital 
data and the more working procedures are digitised, the more qualified and quicker the 
relevant processes become. 

Further mechanisms are for example sunk costs, which refer to the value of previous 
investments in the progress towards digitalisation: Even if a change back to analogue 
processes was possible and knowledge gained from digital processes could help optimis- 
ing analogue procedures, all previous investments in the digital transformation would 
have to be considered as exchange costs. These sunk costs manifest the digitalisation. In 
parallel, interfaces to other organisations within the organisational field are generated, 
which can trigger them to digitalise their organisational procedures or at least the inter- 
faces to link processes between the organisations. Thus, the other organisations need to 
invest in relevant infrastructure as well. 

A higher degree of digitalisation implies that agents work in or with the process of 
digitalisation more and more, developing the process by making alike decisions, and de- 
veloping routines related to the digital transformation. Thus, applying path dependence 
theory to individual agents’ behaviour, the path dependence of the process of the digital 
transformation shapes the behaviour of the affected agents by increasing their tendency 
to show following behaviour and reducing their perceived degrees of freedom. For these 
agents, supporting the digital transformation seems increasingly without alternative. 
The more digitalisation is applied in more and more parts of life, the more the tendency 
to reach decisions to act by merely showing following behaviour is extended to more and 
more areas of life as well. Additionally, the more parts of life are affected by following 
behaviour the more routines develop to reach decisions by following behaviour. Accord- 
ingly, agents are increasingly likely to apply following behaviour in other parts of life as 
well, which do not even need to be directly affected by digitalisation in the first place. 

In general, the digital transformation supports decisions by following behaviour 
because for many topics aggregated data of the masses in form of ratings, quotas, likes, 
or clicks are available. Furthermore, channeled information can be consumed via online 
networks reducing the necessity to review them separately and sort them individu- 
ally within the egocentric PDSN. For example, software information can directly be 
forwarded by a computer expert of a given individual agent or evaluated positively in 
online media, so that there is neither need to raise a question individually nor to actively 
consult the computer expert. Newsletters can function in the same way. But information 
that is dispersed via the social networks can be shaped and received more individually 
because the agent who follows another one is interested in the other’s information and 
evaluation. 

The more an agent involved in the digital transformation is actively present in social 
networks, the more his user profiles map his PDSN. The reason is that when showing 
following behaviour, it is useful to have the critical information right at hand, if neces- 
sary. Accordingly, it is helpful to always maintain mobile contact to these individually 
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important sources, if needed. For example, someone driving a car can use a navigation 
system or if the navigation system is not available, call a friend who knows her way in 
that respective area, or someone with current access to a navigation system. Thus, via 
a messenger service he can ask for an expert or for the directions. Other options such 
as using a street map, or buying one at a gasoline station, or looking at the regional 
map of a bus stop become less and less likely. Based on the individual agent’s routines, 
somebody who is informed predominantly digitally is likely to prefer digital visual or 
digital audible information. 


3.3 Path Dependence, the Digital Transformation, and Networks 


Networks can have different meanings: linked computers (IT-networks), connected 
social agents, data collection networks, analyses of interconnected data, interconnected 
analyses of interconnected data etc. Having these numerous applications of networks in 
mind, the relevance of networks in combination with path dependence and the digital 
transformation can be analysed from the macro level, the micro level, and the meso 
level perspectives. 

In the path-dependent process of the digital transformation, computers or similar 
digital devices are increasingly connected to allow electronic exchange of data to work 
on projects together or to jointly use specific data. With increasing digitalisation, data 
collecting sensors can be linked to each other as well, so that interlinked data can be 
collected synchronically in different places at the same time. Combined data packages 
can then represent a specific point in time making separately collected time series from 
different sources obsolete. Such connected collections of data allow for prompt compar- 
ative assessments that make it possible to immediately detect and respond to variance 
in data points that can point to special events or measurement errors. If necessary, 
survey points can be checked and repaired or readjusted at an early stage. If there is a 
large amount of interlinked data, a team can work together on the same database from 
different locations, utilising interlinked assessments. That way, the inherent advantages 
of different locations can be used at the same time on the same dataset with regard to 
computing power, personal expertise, or sectorial insights. A core feature is that net- 
working increases the work speed: The better the network, the quicker information or 
data can be exchanged. 

For an individual agent who is involved in the path-dependent process of the digital 
transformation, electronic networks imply a quick, prompt, or temporally indepen- 
dent option to download information or exchange data. Accordingly, an agent with 
the tendency to show following behaviour can potentially use diverse options to find 
blueprints for his decisions to act. Nevertheless, at the same time, path dependence in its 
self-reinforcing process increases the likelihood of using practicable (available and not 
completely mismatching) blueprints again and again with the tendency to lock them in. 
Consequently, path dependence manifests or even reduces the number of used sources 
over time (Kominek, 2012; Link, 2018, pp. 38ff., 122ff.). In this sense, the number of 


63 


I Humans and Digital Technology 


sources of such blueprints tends to increase only indirectly again such as when one 
ofthe used sources expands its number of sources or via special recommendations of 
the sources used to actively add the recommendation to the draft portfolio of possible 
actions ofthe considered agent. Thus, additions to the portfolio ofaction blueprints can 
be referred to as closing “structural holes”. These new links are rather an increase in the 
density than a real expansion of a network. 

Considering the coupled PDSNs of agents involved in the digital transformation, IT 
networks allow for higher frequencies of reaction due to the quicker information and 
data exchange. That way, IT networks enable and accelerate cascades of information or 
action dynamics via these coupled PDSNs. One example of such a cascade is the effect 
of the Fukushima earthquake on the promptly following phase-out of nuclear power 
in Germany” followed by the political manifestation and the effect of the victory of 
the Green party in a regional election (Kominek & Scheffran, 2012; Link, 2018). Via 
coupled PDSNs, information of real events can diffuse and trigger other real events in 
totally disconnected regions and with different actors involved who are likely to not 
even know the agents originally affected by the first event. As long as data and infor- 
mation can be received and processed regionally independently via networks, the mere 
information about events can trigger real-life social dynamics, channelled via PDSNs. 


3.4 Path Dependence, the Digital Transformation, Networks, and Al-tools 


The stronger a given network, and the more intensive the analysis of linked data, and 
the quicker these data are analysed right after their collection, the more computer power 
is used in comparison to manpower. To achieve this speed, precoded procedures are 
developed such as neural networks that allow a quicker coding of input for a large 
computing capacity. Software routines are trained by machine learning to process more 
and more data scales or data types quickly to make it possible to promptly assess new 
data right after their collection, e.g. in analyses of X-rays or CT in medical science or a 
translation software in linguistics. Other software is specialised in dealing with very big 
data, even though such handling might be more time-intensive. Examples are meteo- 
rological or climate models. Also, there are deep learning algorithms that can use new 
data to automatically re-optimise the existing software or the application tools generated 
from machine learning. 

The self-reinforcing process of the digital transformation more and more necessi- 
tates the utilisation of so-called Al-tools to enable the handling and use of digital big 
data in real time or with a certain data depth. The potential availability of such tools for 
analyses and the potential transfer to other applications of these tools can also expand 
the volume and intensity of digital data collection and therefore further promotes the 


13 Path dependence reiterates the existing structures. In contrast to Germany, the reaction in France was 
to rather extend the use of nuclear power and to increase funding for research to make the handling 
and operation of nuclear energy even safer. 
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digital transformation. Accordingly, the tendency towards a lock-in in the digital trans- 
formation is connected to the tendency towards a lock-in in the use of these Al-tools for 
analysis because otherwise digital big data would be too big to handle. Analysing con- 
nected digital data is also quicker if AI-tools are used than if they were merely assessed 
by humans. Additionally, AI-tools are applied to discover particular clusters in data and 
thus, raise the connectedness of the data that are analysed. Therefore, AI-tool usage is a 
self-reinforcing process with the tendency towards a lock-in, thus it is path-dependent. 
At the same time, the path-dependent process of AI-tool usage is closely coupled with 
the path-dependent process of the digital transformation itself because each of the two 
enhances the other. 

Agents who are substantially influenced by the digital transformation also heavily 
rely on the output of Al-tools if they receive them via their sources of actions (PDSN). 
Moreover, for each agent a dataset can be created consisting of all his digital activities 
and contacts as well as his real-life activities and contacts that are documented digitally. 
Based on all those data, there are Al-tools that can suggest further digital contacts, 
activities, or even text parts e.g. for E-mail messages. Further digital activities by these 
agents or direct reactions to the suggestions of the respective AI-tools can help to train 
and improve them even further. Suggestions for supplementary activities like advertise- 
ments, videos, news headlines, other web links, or even the writing of another message 
are not chosen like literature resources in the appendix of a text. In order to make agents 
spend more time online, AI-tools rather extrapolate from the developed patterns based 
on profile interests, search history, consumer behaviour, previous journeys, or likes and 
dislikes shared within social networks or social media. Agents who are used to show a 
particular following behaviour have the tendency to follow the suggestions of AI-tools. 
In this process, it can even happen that an Al-tool suggests blueprints for actions that 
the agent would not otherwise have considered at all such as watching more videos even 
though the agent initially wanted to watch only one very specific video. 

Whenever PDSNs are digitally available, they offer AI-tools the chance to link any 
available profile data and the associated interests they determined - such as anxieties, 
likes, relevant topics, or wishes of an agent - with his sources of action blueprints and 
to analyse the resulting network. This way, changes and dynamics of these changes with 
regard to the respective topic or focus can be analysed and potentially it is even possi- 
ble to determine which source in the egocentric network of the agent triggered these 
changes. Diffusion dynamics in PDSNs can be monitored via topic related key words, 
likes, or clicks. Thus, cascades in connected PDSNs can be recognised and analysed, 
as long as Al-tools have access to the specific data for their analyses. For instance, an 
Al-tool reported the observation of the diffusion of a specific illness in China at the end 
of 2019 as part of an early warning system of the WHO (Bluedot, 2020; Niiler, 2020). A 
committee of experts carefully examined the data and initiated additional observations 
to decide which measures should be taken, whether further data would be needed, if 
action should be deferred to until the dynamics of the outbreak provide further insights, 
or if other nations or organisations needed to be warned. 
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In addition to monitoring the altered dynamics within a PDSN or observing real-life 
dynamics by monitoring information cascades in connected PDSNs, it is also possible 
to monitor cascades that spread through real-life events and digital information via 
connected PDSNs. One example of digital reports about real-life events that in turn 
can be analysed as the result of an informational cascade includes the platform TikTok. 
An information cascade triggered and enhanced via TikTok substantially impacted an 
event of the US campaign trail in 2020, which again was subject of many other digital 
reports and news spread via social media (Tagesschau, 2020). 


3.5 Analysis, Options for Intervention, and Final Remarks 


Based on the definition of path dependence as a self-reinforcing process with the ten- 
dency towards a lock-in, it can be deduced by using insights from Social Psychology 
that agents affected by path dependence develop the tendency to show some kind of 
following behaviour. So far, it has never been measured how strong this tendency ac- 
tually is, depending on the kind of influence of path dependence. Sociological studies 
show that routines make up the largest part of our daily activities (Giddens, 1984; Link, 
2018, p. 186). Furthermore, there are statistics that document the increase in average 
time spent online with the ongoing proceeding of the digital transformation. In general, 
suppliers of search engines, social media, or social networks are now able to store big 
data that can be analysed in a short period of time with an ever increasing quality of 
Al-tools. Influencers, followers, likes, and tweets are only examples of digital tools that 
incorporate and document following behaviour. 

Even though it is possible in theory that path dependence can diffuse cascades via 
PDSNs, it can be observed that IT networks have even increased the likelihood of future 
cascades. In the general process of the digital transformation, PDSNs and cascades can 
be increasingly observed and documented digitally. Additionally, the path-dependent 
characteristic of the digitalisation increases the tendency of the involved agents to use 
PDSNs as basis for their actions. The path-dependent process of the development of AI 
tools is entwined with the path-dependent process of digitalisation. AI-tools are increas- 
ingly used and their necessity even grows concurrently the larger the interconnectedness 
and the amount of digitally available data become, which again increases within the 
self-reinforcing dynamics of the entwined path-dependent processes. 

Generally, path dependence describes a reinforcement and manifestation of pro- 
cesses and behaviour. The manifestation implies that path dependence reduces the 
ability to adapt to new situations”, i.e. to situations, in which current behaviour does 
not work anymore or even has counterproductive effects. 


14 In contrast, adaptation to small deviations can occur much faster than without path dependence, since 
the existing path-dependent processes and the hierarchies that result from following behaviour allow 
for a quick diffusion of the small changes. 
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One example is anthropogenic climate change. The particular characteristic of this 
process is that global effects of human behaviour are not automatically compensated 
by regional differences. Instead, all over the world consumption behaviour is based on 
industrial products, which generally produce greenhouse gas emissions during the pro- 
cesses of production, logistics, and even still do so during use of the product. All these 
accumulate in a total global effect. Global warming is the result and causes the necessity 
to change human behaviour: First, to avoid further global warming, second, to reduce the 
potential effects of global warming, and third, to adapt to the changing or future climate. 

If human behaviour needs to be altered at a large scale - either by mitigation or adapta- 
tion - it might be most efficient to start to change action schemes of unintended effects as 
early as possible to efficiently reach a quick and substantial effect. Unfortunately, path de- 
pendence often restrains us from quick adaptation to new situations because in situations 
of lock-ins changes happen only incrementally (if at all). Real change is not impossible 
but it can be that alternatives to the customary are not recognised. Also, the stabilising 
mechanisms of the path-dependent processes rather block any tiny attempts of deviation. 
Thus, in order to achieve a big change of a path-dependent process it is important to 
analyse, which mechanisms self-reinforce the path-dependent process, which underlying 
logic is responsible for the continuity (Tab. 3.1), and what kind of interventions could 
destabilise, dampen, or change the dynamic towards a sustainable change of the process. 

Recall that path dependence also encompasses the tendency to following behaviour. 
Some rather like to follow individual experts, i.e. their “neighbours”, which resembles 
swarming behaviour, while others rather follow the masses, e.g. via information on 
ratings, and show herding behaviour. Once action schemes are set in a path-dependent 
process, it is not always easy to classify them as herding or swarming behaviour. But in- 
dependent from specific processes, social dynamics can be recognised and can become 
societally relevant such as in flashmobs, hypes of speculation, e.g. on crypto-currencies 
like Bitcoin or high-risk stocks like GameStop, or in the US real estate bubble trigger- 
ing the global financial crisis in 2008/2009 (Brunnermeier, 2009; Stein, 2011). These 
can start out by swarming behaviour initiating an exponential increase in participants 
triggering further dynamics. Once a critical mass is reached, people with an affinity to 
herding behaviour join in to cause an additional jump in the number of participants. 
Because of the large number of overall participants, even socially more distanced people 
practicing swarming behaviour can be reached, which further intensifies the cascade 
through the PDSNs and the associated dynamics (see the simulations in Link, 2018). 

Apart from computer simulations, these dynamics of following behaviour result- 
ing from path dependence have not been analysed empirically yet. Thus, no verified 
mechanism of intervention has been identified yet. Currently, many governments try to 
suppress unwished demonstrations or spontaneous large group events by blocking social 
communication media. In some situations, missing communication means cause diver- 
sification and therefore reduce the intensity of the dynamics (such was the case when 
access to Twitter or Facebook accounts were blocked during the siege of the US Capitol 
Building on January 06, 2021. However, in other situations, exactly that kind of shut- 
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Tab. 3.1: Overview of mechanisms capable of producing path-dependent continuity (cited from Beyer 


2010, Table 1) 


Mechanism 


Continuity-ensuring logic 


Destabilisation options 


Increasing Returns 


Self-reinforcing effect 


Formation of adaptive expectations against established 
institutions; transaction costs of change are low and/or 
assessable; transgression of quantitative thresholds in 
combination with substantial efficiency gaps; transition to 
decreasing returns owing to change in the environment 


Sequences 


Irreversibility / Quasi-Irrevers- 
ibility of event sequences 


Overlay of effects; countersequences with annulling 
effect; termination of “reactive” sequences as soon as 
alternative options for action emerge 


Functionality 


Purposes, systemic 
requirements 


Change of functional requirements caused by external 
circumstances; dysfunction as a result of functional com- 
pliance; emergence of significant “secondary effects”, 
replacement by functional equivalents 


Complementarity 


Interaction effect 


Domino effect following partial change; end of com- 
plementarity because of intervening factors; loss in the 
complementarity effect’s relevance 


Power Power saving, power of veto Formation of countervailing power; infiltration or 
“conversion”; influence or “layering” that suggest 
supplementation 

Legitimacy Belief in legitimacy, sanctions Diverging interpretations and traditions; delegitimisation 
because of contradictions, e.g. inexpedience 

Conformity Exoneration from decision, Assertion of a new key conception e.g. because an inno- 


mimetic isomorphism 


vation or crisis calls the old key conception into question 


down of communication access led to an escalation of the situation because the reduced 
ability to communicate via technical devices caused an increased desire for meetings 
in person. This rather reinforced the evolving dynamics with powerful results such as 
the unrests in the Arab Spring, when demonstrations on Tahrir Square in Cairo, Egypt, 
caused the resignation of the Egyptian president (Hanska Ahy, 2016; Tufekci, 2018). 

Theoretically, it can be deduced that there should be a possibility to prevent the 
occurrence of intensive following behaviour that is induced by path dependence in 
advance. To achieve this goal, it is necessary to e.g. 


e reduce the number of path-dependent processes, in which the respective agents 
are involved; 

e persistently try to maintain and point to alternatives and actively push people to 
search, keep, and deliberately choose and use alternatives; 

e sustain variety; 

e bealert, when the phrase “there is no alternative” appears and try to identify the 
relevant path-dependent processes, analyse and, if necessary, reduce their power 
by interventions in the mechanisms of continuity; 

e let people actively and as objectively as possible analyse their decision-making 
processes and question retroactive rationalisations, if necessary; 
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e develop, maintain, and push to apply individual skills to select; 

e inform people who are affected by path dependence about stabilising mecha- 
nisms and the path-dependent process so that they do not generalise their pro- 
cess-related following behaviour. 


Path dependence and the resulting following behaviour can be efficient because deci- 
sions can be made quickly and generally, at least with acceptable quality. Therefore, an 
outsourcing of decision-making processes can be explained by the assumption that the 
person whom an agent follows has access to superior information. Furthermore, indus- 
trial standards, legal texts, and the constitutional pillars of a society form the foundation 
of smooth co-operation and allow a focus on new developments, based on the knowl- 
edge and decisions of previous generations. Substantial social following behaviour can 
include the sentiments of people who are generally less politically involved in daily pol- 
itics, which shapes the agenda, and increases political sensitivity for societal problems. 
Likewise, substantial following behaviour can be problematic in democratic decisions: 
If each voter just regurgitates the political tenor of the person he or she follows, political 
decisions merely count majorities of polarised dynamics, regardless of any variety in 
voter’s livelihoods. Accordingly, Facebook has noted that in many nations the candidate 
with the largest group of followers on Facebook has won the election in the recent past 
(Zuckerberg, 2017). 

With regard to the digital transformation, IT networks, use of Al-tools and path 
dependence, this chapter has clearly laid out that the digital transformation can be 
explained as a path-dependent process that is stabilised and reinforced by IT networks. 
Also, the use of AI-tools as well as IT networks increases in the self-reinforcing mecha- 
nisms of digitalisation. In this current exponential growth of digital processes, it appears 
to be mandatory to encourage dynamic participation. At the same time, the expected ef- 
fects are often described as disruptive, a notion that is supported by various perspectives 
presented in this book. It can be expected that the following behaviour of individuals 
is increased by digitalisation and particularly by the intensive use of information and 
communication technology. Simultaneously, global networks can lead to a globalisation 
of potential polarisations and contextual dynamics. Thus, current political agendas are 
shaped by connected dynamics” instead of by local societal problems. Then democratic 
decisions only represent a snapshot of the local share of potentially interconnected 
polarised dynamics. In these dynamics, the contents as well as the underlying societal 
forces can be controlled by AI-tools if voters use social media intensively. 

At the beginning of a path-dependent process it is not possible to predict, at which 
level the lock-in eventually sets in, according to Arthur (1994) and the urn model and 
others who emphasise the contingency characteristic (Vergne & Durand, 2010). How- 


15 Accordingly, there were assessments whether the British people had indeed opted for Brexit, had the 
voters optimised their own preferences and thus reached a rational decision. Instead, social dynamics 
prior to the Brexit vote might have actually caused the majority of the voters to favour the British exit 
from the European Union (BBC, 2016; Clarke, Goodwin, & Whiteley, 2017; Usherwood, 2017). 
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ever, a classical urn model describes a process without intervention. In contrast to a ball 
in an urn, a human being can theoretically decide and act differently from what mere 
following behaviour would suggest. 

Comparing countries, current levels of digitalisation vary substantially and there 
are parts of everyday life that are digitalised in some countries but not in others. An 
intensive use of smartphones to stay in touch with social networks is already common, 
which creates strong dependencies (Torevell, 2020). In Germany, for instance, there 
is currently an increasing tendency to include the use of smartphones in numerous 
daily activities to an even greater degree. The use of smartphone apps e.g. representing 
the personal identity card are encouraged and standards are set that make credit card 
payments without smartphone use substantially more difficult. 

In contrast to the exponentially increasing majorities in an urn model, polarisations 
can reinforce both oppositions in real life. Thus, if there was strong opposition to the 
general digital transformation that discusses and preserves alternatives, e.g. to maintain 
flexibility for future activities, it could be perceived that the dynamics of the digital 
transformation changed from self-reinforcing exponentially to consciously reflected and 
in some application areas linearly instead. This change in dynamics could help societies 
to have more time to actively manage changes, prevent collateral damage, and reduce 
the disruptive consequences of adverse side effects. It may be possible to influence which 
topics should be connected to or disconnected from the exponential growth dynamics 
of the digital transformation to benefit economically, maintain scientific excellence, and 
preserve geostrategic autonomy. In this strategic and more proactive approach there 
should be a public debate, which parts of social life should perhaps actively remain 
exempt from the digitalisation process in order to assure the recreation and preserva- 
tion of individual decision-making competencies and the functionality of democratic 
processes. 
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Questions in the Philosophy of Technology 


Stefan Bauberger’® 


Value-free Technology? 


A naive understanding of technology considers it as a tool that is intrinsically value 
neutral, but which can be used for good or for bad. Therefore, it is the use alone that 
determines the value of technology. A hammer can be used to hammer in a nail, but 
also to smash someone’s head. 

Behind this image is the correct intuition that technology does not have an end in 
itself but that it should serve people. However, a differentiated view shows that this 
intuition is not a description, but that it defines an ideal, and thus a task, namely the 
task of preserving this service character of technology. 

Some techniques already carry the tendency towards a certain application. A nuclear 
bomb will never be used directly for good, although even there, good applications are 
conceivable. One such good application would be the utopian scenario of deflecting 
a large meteorite on a collision course with Earth from its path by means of a nuclear 
bomb. Even if such a scenario is conceivable, one still cannot meaningfully speak of the 
development of the nuclear bomb as being value neutral. 

A reverse consideration can be made for the development of effective and affordable 
medicine, although these can be abused, too, for example, by being embedded in an un- 
just health system. However, one can assume that such medicine represents an example 
of technology that is usually good in itself. 

In most cases, digital technologies are indeed ambivalent in the sense that their 
value depends on their application. However, the boundaries between development 
and application are fluid. An example of this is the development of artificial intelligence 
(AI) for military technology. These developments are not only further developments 
of already existing technology, but entirely new and frightening dimensions of war 
technology can be expected if these developments are not stopped by international 
agreements. Automated aerial drones, for example (which paradoxically become more 
dangerous the smaller they get), can be used for targeted killing or for acts of sabotage 
against infrastructures, whereby the attacker does not run any risk of his own and may 
not even be identified. They thus delimit warfare and lower the threshold for acts of 
war. In the 10th chapter, Götz Neuneck makes this forcefully clear. 


16 Cf. with other chapters by the author: Bauberger 2020a and Bauberger 2020b. 


71 


I Humans and Digital Technology 


Greater Impact Through Embedding in Social and Economic Contexts 


Furthermore, it should be noted against a naive application of the tool metaphor that 
the effects of technical developments are often - and this is to be expected in connection 
with digitalisation and AI - not limited to the concrete individual case of application, 
but that they must be considered in relation to their overall social and economic impact. 

The development of the steam engine in the 18th century and about 100 years later 
the technical utilisation of electricity and the invention of mass production with the 
assembly line each resulted in huge social upheavals, with economic winners and los- 
ers. Even more powerful was the effect of one of the oldest technical inventions, that 
of agriculture about 10,000 years ago. Only then dense settlements were possible, only 
then cities could arise, only then individuals or social groups could accumulate great 
wealth and so on. 

An example of the relevance of this embedding in larger contexts arises - as with 
the earlier industrial revolutions - from the effects of digitalisation on the economy 
and the labour market. It is no coincidence that new monopolies and forms of platform 
economies have emerged in a very short time with the large digital corporations. The 
very low marginal costs in the application of digital technologies, as well as “positive” 
feedback through network effects, favour a “the-winner-takes-it-all” economy. 


4.1 Technology, Modernity, and the Image of Man 


The modern image of man is in many respects shaped by the natural sciences - that is, 
the widespread view. Naturalism, materialism, physicalism, and reductionism go hand 
in hand with the natural sciences, or at least they become a constant challenge to every 
image of man. 

These positions are philosophical views. They are not directly the result of scientific 
knowledge. However, they are usually seen as a consequence of the natural sciences in 
the history of thought. The proponents of naturalism understand their view as a con- 
sequence of the results of natural science insofar as, according to their understanding, 
this provides a complete explanation of the world, including the position of the human 
being. The human organism is understood mechanically. The philosopher La Mettrie 
coined the expression “L Homme Machine” as early as the 18th century (De La Mettrie, 
1748). 

From a philosophical perspective, these naturalistic and materialistic positions can 
be understood as absolutisations of scientific knowledge. They derive a claim of sole 
representation from the success of the natural science programme. Natural science is 
not based on pure objectivity, but is the result of an objectification that includes many 
preconditions that cannot be derived from natural science itself. In concrete terms, sci- 
entific knowledge always rests on the background ofa life-world knowledge that is much 
broader, and which can never be completely objectified, but which must be presupposed 
for natural science (cf. van Fraassen, 2002 and Bauberger, 2011). 
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Ernst Kapp, a pioneer of the philosophy of technology, understood technology as 
an interplay. On the one hand, technical products are an “organ projection’, i.e. an 
extension of the natural abilities of humans, shaped according to the pattern of these 
abilities. On the other hand, man understands himself anew in this projection (Kapp, 
1877)", according to the model of his own creation. Technology therefore also shapes 
the image of man. Man’s self-projection detaches itself from him as an image that, as a 
technical creation, is more perfect than he actually is in the respective area. And then, 
man understands himself from the image of his perfection in technology. However, this 
perfection is always only a partial perfection that picks out and enhances certain abilities 
of the human being. In this way, man’s self-image is accentuated anew in the direction 
of this ideal that technology presents. 

In this view, the materialistic and mechanistic world view can be understood as a 
consequence of mechanisation. In particular, the development of the steam engine and 
other prime movers (motors) clearly showed that movements can be caused purely ma- 
terially. Through digitalisation and especially through AI, a new image is being formed: 
Man as an information processing system. Floridi, one of the leading digitalisation phi- 
losophers, speaks of the “infosphere” as a new model of human reality: “The next step 
is to rethink in terms of reality and to rethink more and more of its aspects in terms 
of information.” (Floridi, 2015, p. 75) He speaks of a “dematerialisation of objects and 
processes.” (Floridi, 2015, p. 101)'® Weizenbaum and Janich pointed out some time 
ago that this is philosophically based on a hypostatisation and naturalisation of the 
concept of information that is factually incorrect (cf. Janich, 1997 and Weizenbaum, 
1977, pp. 207-241). 

Parallel to this, the penetration of the living world by computer technology leads 
to the cyber world being seen as the realisation and transcendence of what the human 
being actually is. Material reality is surpassed by an apparent spiritual reality. Wiegerling 
warns in this context of a “loss of resistance” (Wiegerling, 2011, pp. 26ff), due to the 
“loss of reality” (Wiegerling, 2011, p. 13). 

The consistent exaggeration of this ideal leads to radical transhumanist fantasies. Ray 
Kurzweil, head of technical development at Google, developed the vision that robots 
and computers will eventually take control of the world, as they will overtake humans 


17 A still current example of this retrograde projection is the understanding of the heart as a pump (cf. 
Kapp, 1877, 98), i.e. modelled on the basis of mechanical pumps. I owe the hint of the significance 
of this retrograde projection on the self-image of the human being precisely connected to computer 
technology to Eugen Wissner, in the context of a master’s thesis he wrote at the Munich School of 
Philosophy. 

18 Further quotes from Floridi to clarify: “According to the ‘it-from-bit’ hypothesis, even our bodies are, 
deep down, (composed) of information, not of some ultimate material substance distinct from what 
is immaterial.’ (Floridi, 2015, 101) 

On privacy: “... so too privacy calls for a correspondingly fundamental reinterpretation that takes ac- 
count of the fact that our selves and our interactions as inforgs are informational by their very nature.” 
(Floridi, 2015, 159) 
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due to the progressive development of AI (Kurzweil, 2015). In his sense, this is not a 
loss, but it is in the good logic of evolution. Behind this is the idealisation of intelligence 
in the sense of data processing, and subsequently the reduction of humans to the same 
function. 

The effects of digitalisation on the image of humans are particularly relevant, for 
example, in the field of medical technology and care for the elderly. In these areas, there 
is a high cost pressure (due to the politically set framework conditions), which elevates 
efficiency increase to the ideal. In addition, there is also a tendency towards mechanisa- 
tion independent of digitalisation, which competes with the ideals of medicine and care 
centred on people. However, especially in care, people must be accepted in their bodily 
dimension and with their limitations, even with mental limitations such as dementia. 
If being human is defined according to the ideal image of an information-processing 
system, people in need of care can only be understood and treated as annoying marginal 
phenomena of society. 


4.2 Mortification, Alienation, and Fascination 


Günther Anders has described the human and social consequences of the development 
of technology as a sequence of ever-increasing mortifications and alienations (Anders, 
1957, and Anders, 1980). Measured against the perfection of technology, for example, 
the power that machines unleash, humans experience themselves as imperfect and pow- 
erless. The nuclear bomb in particular demonstrated this powerlessness in a special way. 
Accordingly, in his philosophy of technology, Heidegger also criticises an alienation of 
man that arises from the power of technology detached from him (Heidegger, 1954). 

The antithesis to this mortification is the fascination that comes from highly devel- 
oped technology. The good thing about this fascination is that researchers drive the 
development of technology with great dedication. On the other hand, the fascination 
with technology carries the danger that technology becomes an end in itself. 


4.3 Technical Solutions for Human Problems 


“Instrumental reason has turned words into a fetish surrounded by black magic. And 
only the magicians have the rights of the initiators. And they play with words and deceive 
us” (Weizenbaum, 1977, p. 334). This is how Joseph Weizenbaum, one of the great 
pioneers of AI research, criticised the exaggerated hopes placed on computers as early 
as 1976. He pleads for a “reason that recalls its human dignity, authenticity, self-respect, 
and individual autonomy” (ibid.). 

Weizenbaum criticises a paradigm of technical reason that combines with and abso- 
lutises scientific rationality. In this paradigm, problems are there to be solved, and they 
are ideally solved technically. This paradigm goes beyond the area of the application of 
technology; it has become independent in this respect. Weizenbaum cites as an example 
that in the American student riots, it was often suggested that the problems should be 
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solved through better communication between the different parts of the universities. 
“But this view of the ‘problem’ (...) effectively hides and buries the existence of real 
conflicts” (Weizenbaum, 1977, p. 328). This “imperialism of instrumental reason” (Wei- 
zenbaum, 1977, p. 337) misses reality, or rather reduces this reality to everything that 
can be technically described and solved. Humanity is thus lost. In the human realm, 
many things are complex; even problems cannot be solved in many cases but must be 
endured. Human support is important far beyond the solution of technical problems. 

Under the premise that technology should serve the well-being of humanity, the 
conclusion of the considerations in the philosophy of technology is that this tool char- 
acter of technology is by no means a given. Instead, it is a permanent challenge to make 
technology serviceable. As a rule, but not always, technology is ambivalent. However, 
the use of technology must be considered in the social and economic context (with the 
associated interests), the danger of technology becoming an end in itself must be taken 
into account, as well as the effect on the image of man. 

Again, the use of digital technology in medicine can be used as an example, espe- 
cially if AI can take over many diagnostic tasks in the future. In the ideal case of good use 
of AI, the technical tasks of the medical art are transferred to technology, freeing doctors 
for the tasks associated with the human aspects of their work: as caring, compassionate 
people who, for example, discuss therapy options with patients, whereby decisions are 
made not only according to purely technical aspects, but also take into account the 
values and attitudes to life of the patients, as well as their environment. In the worst 
case, digitalisation reinforces the technical character of medicine. Economic constraints 
and lobbying interests within the health system will promote this bad case. In addition, 
there is the strongly technical image of man within this system (man as a biochemical 
machine) and the associated paradigm of technical reason. 

Only with knowledge of these mechanisms can we succeed in turning the applica- 
tion of these ambivalent AI techniques, which also open up great new possibilities for 
medicine, to the good. For the good, these techniques promote the humanisation of 
medicine, so that the human and interpersonal aspects of healing gain value as more 
technical aspects of theories are transferred to machines. 
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Digital Extensions, Transhumanism and Posthumanism 
Frank Schmiedchen 


The chapter follows up on the previous one and deals with our self-perception and 
the image of humanity with regard to the topic of the book and how human beings 
‘should be in the future: It asks to what extent digital technology implanted into the 
human body can lead to changes of the person in essence, so that this technologically 
“upgraded” human being” becomes “a different being” (e.g. a cyborg) and what moti- 
vates such self-improvements. This is followed by a look at the essential statements of 
transhumanist and (technological) posthumanist ideas, also with regard to underlying 
images of the world and the human being. Subsequently, reflections on the intangible 
dignity of every human being are presented as counter-concept, and the various ap- 
proaches are discussed. 


5.1 Forms of Digital Self-Improvement 


In the context of this chapter, digital enhancement is understood as digital technology 
permanently connected to the body to improve, support, or restore knowledge, skills, 
or abilities. Not all forms of digital self-improvement are relevant to the question of to 
what extent they trigger changes in human identity and personality. In particular, all 
neuroimprovements may have substantial effects on the consciousness and personality 
of the improved people and bear a relevant risk of being hacked and used to manipulate 
or control the human (cf. Clausen, 2006, p. 28). 

Such improvements can be caused by assistive devices?°, prostheses”, and neuroim- 
plants as well as genetic and nanotechnological changes in humans.?? Above all, neu- 
roimplants to support cross-task skills, knowledge acquisition, and processing must be 
examined in principle for potentially triggering psychological, cognitive, and mental 
changes and having strong ethical implications (cf. Fenner, 2019, pp. 167-288). Such 


19 In the rest of the text, the inverted commas are omitted from words such as improvement, extension, 
etc. for the sake of text flow, even though the author is skeptical if they are improvements and who 
measures such improvement and how. 

20 They are often classic and analogue products replacing, supporting, or relieving human performance 
to prevent or compensate disabilities or to treat illnesses (e.g. visual and hearing aids, body replace- 
ments/ prostheses, orthopaedic aids, incontinence and stoma articles, and technical products that 
serve to introduce medicine or other therapeutic agents into the human body. 

21 While exoprostheses are body replacements that are attached but outside to the body (e.g. replacing 
limbs), endoprostheses are artificial materials implanted to the body. 

22 Non-digital genetic or nanotechnological enhancements raise analogous questions, but these are not 
discussed here. 
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neuroimplants are human-machine interfaces (brain computer interfaces, neurolinks) 
that directly connect human nerves with electronic technology and serve to exchange 
electrical signals between the brain and the respective technical devices (cf. Clausen, 
2011, p. 3; Clausen, 2015, pp. 697-839; Jansen, 2015, pp. 226-234; Blumentritt/Milde, 
2008, pp. 1753-1805). For example, performance-enhancing brain implants are used 
to improve memory performance. One or more interfaces implanted in the brain are 
connected to a chip in the brain or outside and are supposed to potentiate cognitive 
abilities. A distinction is made between (passive) deriving and (active) stimulating sys- 
tems, whereby modern research assumes that we will soon arrive at integrated systems 
that serve both aspects (cf. Müller/Clausen/Maio, 2009). 

In view of the scope of these interventions, the question must be asked whether 
the numerous and far-reaching upgrades, especially in the neurological field, are the 
historically contemporary form of human striving for self-improvement and self-con- 
quest (cf. Fuller, 2020, pp. 44-55; Hansmann, 2016, pp. 28-30), or whether the digital 
enhancements available today (or soon) are generating a new quality finally putting into 
question humans as such? This new quality would always be the case when interven- 
tions lead to a fusion of technology and humans creating ‘something new’ by changing 
consciousness and identity leading to transformed creatures (transhumans, cyborgs). 
(cf. Rössl, 2014, pp. 21-28; Jansen, 2015, pp. 219-226; Sorgner, 2018, pp. 90-92.; Loh, 
2018, pp. 58f). 

But also noninvasive neurotechnologies capable to “read the brain and write to it” 
are essential (Schwab, 2018, p. 243). “A more precise influence on the brain could ma- 
nipulate our self-perception, completely redefine what experiences mean, and funda- 
mentally change what reality represents” (ibid., p. 244). 

Enhancements of biological life through digital implants could potentially cause 
a convergence of man and machine that may ultimately lead to an all-encompassing 
network of life and the digital, raising the question of the extent to which such a result 
definitively transcends the boundaries of what it means to be human. This, in turn raises 
the question of whether and at what level there is an insurmountable difference between 
man and machine, and what this difference is based on. 

However, the pursuit of self-improvement can also be very pragmatically motivated, 
free of such profound philosophical reflections, and explicitly or implicitly based on 
utilitarianist ethics. According to this, everything that is useful would also have to be 
ethically compelling. 


5.2 Utilitarian Motivation 


For utilitarianists, digital enhancements are a possible and useful, and thus mandatory 
improvement of the human being in order to reduce or eliminate observable deficits 
of the individual or of humanity as a whole. Only those improvements that benefit 
the individual but harm others or the public are to be rejected (cf. Birnbacher, 2013, 
pp. 153-158). Humans perceive the world, reflect, or abstract these sensory impressions, 
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and then invent ways to better master the nature, in which they live or compensate for 
their deficits in terms of mastery. Humans must therefore complement themselves with 
things they create (cf. Rössl, 2014, p. 15). Humans develop their abilities by inventing 
things that improve their performance. Thus, the weak hand was first strengthened or 
replaced by simple tools (hand axe, sickle) and later by (automated) machines. The 
same applies to visual aids improving the spectrum of individual vision or the human 
ability to see per se. Without a telescope or microscope, humans would not be able to 
see anything far away or very small. Analogue aids, therefore, already fundamentally 
expand the spectrum of human possibilities. 

Since the Renaissance and even more so since the end ofthe nineteenth century, the 
idea of competition has played a central role in these enhancements. People consider 
themselves to be in a permanent competition to see who is more successful, cleverer, 
more beautiful, etc. Today, this basic motive of individualised competition has become 
the source of a permanent sense of deficit due to potentially omnipresent digital-social 
networking. It is a subjectively felt deficit vis-a-vis believed or actual competitors, but 
also vis-a-vis the theoretically maximum possible (cf. Spreen/Flessner in Spreen et al., 
2018, pp. 8-10). 

As already explained in the last chapter, Giinther Anders therefore speaks ofa result- 
ing fundamental mortification of man in the face of the machines he has created. This 
humiliation is the result of a growing gradient between one’s own subjectively perceived 
imperfection and the ever-increasing perfection of the machines (Promethean gradient) 
(cf. Anders, 1956/1980, p. 16). In this respect, digitalised technology is only a further 
step, but as the book shows, also a qualitatively new step on the path people followed 
since they became self-aware. 

Consequently, the optimisation of humans through the implantation of digital 
technology can be interpreted as a product of human invention for humans. From 
a utilitarian perspective, an omission of possible, digital improvements is unnatural 
and unethical, since an omission runs counter to man’s continued, historical deci- 
sions or, more broadly, his natural law compulsion to constantly improve (cf. Nida- 
Rümelin/Weidenfeld, 2018, pp. 64-70). In this understanding, exceptions are only 
permissible where dangers for others or humanity as a whole are deemed too great to 
continue using a technology once it has been invented/discovered. This corresponds 
more or less to the experiences of the 20th century, in which only those technological 
paths were at least partially abandoned that have proven that they can cause mass 
destruction (e.g., toxic gases), or where alternative technologies show fewer side effects 
(e.g., CFCs, nuclear fission). 

The majority of pragmatically oriented self-improvementists will affirm that human 
life should be improved in its natural life span in every possible way. They will probably 
also agree that at the same time, whenever possible, people’s life spans should be ex- 
tended. Thus, pragmatic utilitarians also dream of immortality and do not ask whether 
such a development is desirable, but affirm this assumption without reflection (cf. also 
Fenner, 2019, pp. 148-159). 
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5.3 Transhumanist Motivation 


The most important transhumanists?? go beyond such purely pragmatic understanding 
ofa “better-is-always-useful” utilitarianism. For them, it is a matter of supplementing or 
replacing biological evolution on earth (and in the cosmos in general) by cultural-civil- 
isational-technical means to accelerate evolution and improve the result (cf. extropia- 
nism: Erdmann, 2005). 

The idea of transhumanism (“beyond-human”), which essentially only** emerged 
in the 20th century, aims to evolve homo sapiens sapiens as a species (cf. More, 2013; 
Kurzweil, 2014; Sorgner, 2018). Numerous original voices and manifestos profess this 
common fundamental belief.” Transhumanist concepts share a view in which human- 
ity no longer feels bound to its approximately 300,000-year existing form (cf. Gunz, 
2017). Rather, possible sociotechnical evolutionary steps, including those that lead 
beyond humans, are regarded as naturally human. Thus, the biologically constituted 
human being is supposed to improve himself in every technical form possible to him. 
This explicitly affirms that the result of such a synthesis is in fact another creature 
(transhuman, cyborg) or that the contents of human consciousness only exist in digi- 
talised form; a dream shared by trans- and technological posthumanists. Such a view 
then applies to all technical interventions that humans undertake in order to (co-)shape 
evolution. The nature of these interventions (e.g., genetical, nanotechnical, or digital) 
is irrelevant. The changes stem from human creativity and are therefore natural in this 
view (cf. Sorgner, 2018). 

Transhumanist concepts are not solidified doctrines of ideas, ideologies, or theories. 
Instead, they are diverse and partly contradictory ideas that are based on different ba- 
sic philosophical convictions. However, the spectrum is dominated by strictly atheistic 
technologically oriented thinkers but can also be found in Protestant-Calvinist or Jew- 
ish-derived beliefs. Their common basic belief is that humans are not fully developed 
but in a permanent state of further development, in which they themselves actively carry 
evolution beyond their previous humanity through scientific knowledge and technical 
progress. In this context, for some of them, transhumanism represents a kind of inter- 
mediate stage to posthumanism, so that the boundaries are blurred here. 

Transhumanist concepts are based on a paradoxical image of man, which de- 
scribes human as imperfect and emphasises the necessity for his self-improvement 
and self-conquest, but also sees humankind as the “awakener” of the cosmos who 


23 e.g. Nick Bostrom, Fereidoun Esfandiary (FM-2030), Eric Drexler, Ben Goertzel, Aubrey de Grey, Yu- 
val Harari, James Hughes, Julian Huxley, Zoltan Istvan, Saul Kent, Timothy Leary, Ralph Merkle, Max 
More, Elon Musk, Gerard O’Neill, Larry Page, Martine Rothblatt, Anders Sandberg, Pierre Teilhard 
de Chardin, Peter Thiel, Natasha Vita-More, Ken Warwick 

24 Some transhumanists refer to ancient sources. However, since there is not even now a stringent trans- 
humanist theory, only the period in which most transhumansitic ideas were developed is relevant. 

25 Examples: https://humanityplus.org/; https://humanityplus.org/philosophy/transhumanist-decla- 
ration/; http://www.transhumanismus.demokratietheorie.de/docs/transhumanismus.pdf; https:// 
transhumane-partei.de/was-ist-transhumanismus/; (each 03 January 2021) 


80 


5 Digital Extensions, Transhumanism and Posthumanism 


provides it with a meaning. In this respect, transhumanism is also transcendentally 
oriented and aims at that which lies beyond the human being, but can be reached 
by him (cf. Coeckelbergh, 2018). Man, as a weak and imperfect living being, needs 
constant improvement to be whole. Humans can and should gradually improve their 
genetic makeup, physical and mental health, perception, emotional and cognitive abili- 
ties, and skills, and significantly extend their lifespan (cf. Boström, 2008, pp. 107-136). 
This is achieved through the permanent incorporation of technology (genetic, digital, 
nanotech, etc.) (cf. Hayes, 2014). 

The core element of this conviction emphasises that it is human intelligence, his 
cognitive capabilities, that calls man to higher things. In contrast, biological corpo- 
reality, including emotions, is only an expression of previous inadequacy of “blind” 
evolution up to our present. Nevertheless, all transhumanist concepts are ultimately 
entangled in a paradox of quasi-religious-transcendental and materialist thinking (cf. 
Coeckelbergh, 2018). Ultimately, it is about a physical vision of human expansion, even 
if this is reduced to the brain content, which as the sum of its physical manifestation fully 
constitutes consciousness - the mind. In this sense, uploadable consciousness is the sum 
of available and processed information. Only for this reason can the mind be uploaded 
onto a computer, because for most transhumanists it is materialised information and 
nothing more than that. 

Despite this reductionist, materialist understanding, there is an inherently quasire- 
ligious interpretation of man overcoming himself and only thereby becoming himself, 
but also numerous philosophical and historical points of contact, for example with Prot- 
estantism, especially in its Calvinist and evangelical forms (e.g., currently strong among 
the Mormons) or in Judaism (cf. Samuelson/Tirosh-Samuelson, 2012, pp. 105-132; 
Krüger, 2019, pp. 110-115). Since the Enlightenment, human reason, intelligence, and 
linguistic ability were identified as expressions and signs of man’s calling and capacity 
for higher things. These spiritual qualities indicate that man is the image of God. 

Transhumanists tend to make references to the thinking of Herder, who describes 
humans as deficient beings. For they are the “most miserable of the animals” This results 
in the possibility and necessity for him to constantly improve and elevate himself by 
means of his intellect and language (cf. Herder, 1769, Part Two, First Law of Nature). In 
contrast, it continued to be true, and since the Victorian 19th century even more so, that 
man’s natural physicality and the associated drives (especially sexuality) and feelings 
were the expression of eternal sinfulness and inferiority that needed to be overcome or 
redeemed?® (cf. Lüthy, 2013, pp. 11-25; Mulder, 2013, pp. 30-43; Samuelson, Tirosh- 
Samuelson, 2012, pp. 105-132). 

Then, at the end of the nineteenth century, in the context of the growth of capitalist 
convictions, the Protestant postulate was further developed, especially in the USA, into 
“the-successful-is-successful-because-God-loves-him”. Finally, we also find ample refer- 


26 In contrast, Luther emphasises in his Diputatio de Homine the impossibility of determining reason as 
a defining characteristic of man, since it could just as well be an expression of his “sinfulness”. 
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ences to Nietzsche and his need for the creation of the divine superman. The creation 
ofthe Nietzschean superman advertises a tantalising transhumanist proposition: Ifthe 
first cognitive revolution made the “insignificant African ape” master ofthe world, the 
second cognitive revolution will elevate man to “master of the galaxy’, quite peacefully 
with the help of genetic engineering, nanotechnology, and interfaces between brain 
and computer (cf. More 2010; Harari, 2017, p. 476f.). In its quintessence, this leads to 
a posthumanist view that evolution as a whole should be regarded as imperfect and 
therefore be supplemented or replaced by scientific-technological evolution (cf. Sorgner, 
2018; Jansen, 2015, pp. 219-234). 


5.4 Posthumanism 


Posthumanism, which is also extremely heterogeneous, is divided into a technologically 
oriented majority camp?’ and a critically oriented, small minority camp?®. Apart from 
the relatively uninfluential small group of critical posthumanists, all authors understand 
socio-cultural progress as a consequence of technological developments.”° 

Technological posthumanists argue that the (final) human creation (our final inven- 
tion) in the form of the singular superintelligence and its then “descendants” created 
without human intervention will awaken the cosmos, but not humans themselves (cf. 
Barrat, 2013). Their approach thus leads them to the normative demand of the final and 
complete overcoming of man by his technical inventions, since these are superior to 
him. Accordingly, humanity should make room for a (digital) superintelligence (singu- 
larity) created by it, which, superior to all humanity as a de facto God, creates paradise 
on earth and in the cosmos - at its discretion, with or without human beings. 

Although “real” transhumanists only aim at the improvement of human life on a 
higher evolutionary level and do not want to leave the Anthropocene but extend it, 
many authors who also express themselves as being transhuman are ultimately techno- 
logical posthumanists*°and see humanity only as a necessary intermediate step towards 
higher, nonhuman but human-induced intelligence or life forms that will ultimately 
awaken the cosmos. Without simplifying too much, technological posthumanism can 
be described as a current of thought/quasi-religion that is hostile to human beings. The 
way of dealing with people who actively strive for technological posthumanism should 
therefore be discussed socially as soon as possible. 

In this context, the following is also significant. In the confrontation between “real” 
transhumanists and technological posthumanists, a conflict of epic proportions is per- 
haps brewing, which will not only be fought on paper. In this already emerging, pos- 


27 e.g. Raymond Kurzweil, Marvin Minsky, Hans Moravec, Vernor Vinge, Eliezer Yudkowsky 

28 e.g. Rosi Braidotti, Janina Loh, Stefan Sorgner 

29 https://whatistranshumanism.org/ (02 March 2021) 

30 The boundaries between the two groups can only be determined philosophically-analytically anyway. 
See on this: Loh, 2018 
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sible (hot) war, the transhumanist side fights for the continued existence and further 
development ofa technically improved humanity in the sense ofan expanded concept 
of philosophical humanism, while the technological posthumanists want to overcome 
humanity in an antihumanist way through a new “technology species”. Elon Musk 
explicitly justified the founding of his company Neuralink by saying that humanity 
would have to arm itselffor the inevitable war against an artificial intelligent singularity 
through human-machine interfaces to survive”. 

Besides the technological posthumanists, there is a small group that calls itself criti- 
cal posthumanists with a completely different vision. This group does not strive for the 
technological overcoming of humans, but wants a fundamental ethical human renewal 
by overcoming humanistic, anthropocentric ways of thinking and behaving, with which 
humans have illegitimately elevated themselves to being the murderous master of the 
world and all its life forms (cf. Braidotti, 2014; Loh, 2018). However, technological tools 
are not rejected along the way. What the critical posthumanists have in common with 
the technological posthumanists is merely the demand to overcome the Anthropocene. 
But critical posthumanists only want to push humans down from the throne of the 
crown of creation so that they can consciously integrate themselves into the overall 
organic structure of the planet, in which they are always and completely embedded 
anyhow. This way, socially and ecologically reprehensible actions committed in the 
past and present should be avoided in the future (cf. Braidotti, 2014, p. 197f.). Thus, 
critical posthumanism is a current of the pseudoleft (il)liberalism that we know from 
the identity politics, antihumanist debates. Critical posthumanism is hardly noticed 
in mainstream trans/posthumanist discussions, especially not in the US and the P.R. 
China. Since critical humanism is also not a (primarily) technology-oriented current of 
thought, it is only mentioned in this book for the sake of completeness. 


5.5 Man as a Self-Purpose — Machine as a Tool 


If one wanted to make it (too) simple, the human striving for complete control over 
one’s own life, the environment, and one’s own death could be psychologically described 
as infantile, narcissistic, and characterised by hybrids (cf. Nida-Rümelin/Weidenfeld, 
2018, pp. 188-197). A more serious engagement with the radical views put forward on 
human self-improvement and on trans- and posthumanism* requires a clear vision 
regarding one’s own self-image as well as the underlying image of humanity. 

We contrast the trans-/posthumanist view of human beings, which is only roughly 
sketched here, with a view that values the “imperfection” of human beings and biological 
processes as something natural, intentional, and meaningful. In this view, illness and 


31 https://neuralink.com/; https://www.sciencemediacenter.de/alle-angebote/press-briefing /details/ 
news/brain-computer-interfaces-hintergruende-zu-forschungsstand-und-praxis/; https://industrie- 
magazin.at/a/neuralink-warum-elon-musk-computerchips-direkt-ins-hirn-pflanzen-will 

32 In the following sections, posthumanism always only refers to technological posthumanism 


83 


I Humans and Digital Technology 


death are natural phenomena that are meaningful (cf. Bauberger/Schmiedchen, 2019; 
Coekelbergh, 2013, pp. 21-23). 

Such an acceptance ofthe consequences arising from the complete and indissoluble 
interconnection ofhumans with nature is considered suboptimal by trans/posthuman- 
ists. For them, it is morally reprehensible to resign unnecessarily. For them, a view, in 
which the human being is an integral, indivisible manifestation of the ecosystem earth 
(and the cosmos) in its physical constitution that goes from birth through growth, re- 
production, and decay to death, is an inexcusable capitulation in the face of the assumed 
creative powers of man to change his destiny. 

The religious scholar Oliver Kriiger speaks of posthumanism as the attempt to 
“heal” the four grievances of the human being. In doing so, he refers to the work of the 
philosopher Johannes Rohbeck who in turn based his ideas on thoughts of Günther 
Anders, Hannah Ahrendt, and Sigmund Freud (cf. Krüger, 2019, p. 423; also: Flessner, 
2018). He continues to say that the alternative to these grievances “lies in the uncom- 
fortable acceptance of human imperfection, death, and ageing”. In this sense, it can 
be argued that transhumanist thinking leads to a loss of the specific value of humans 
as biological life form, because in transhumanism “human nature finds a sensitive 
devaluation in its vulnerability, its transience, and above all its replaceability” (Hans- 
mann, 2016, p. 109). We resume here and focus on the self-purposefulness of life (cf. 
Bauberger/Schmiedchen, 2019). In our view, technology always remains instrumental 
(see the following chapter). The usefulness of technology for humans is therefore an 
essential basic condition for the justification of technology. 

Those who do not want to follow our reasoning can help themselves as follows: 
How do transhumanists want to prevent an increasing “inertia” of necessary adaptation 
processes in case of a massive extension of lifespan and the (presumably unrealistic) 
use of cryonics (cf. Fuller, 2020, pp. 184ff.). Sexual reproduction contains a “built-in” 
mechanism for better adaptation in an apparently permanently changing universe. This 
ensures the preservation of the species. Even more mundane is that even the greatest 
minds of humanity get tired over the decades and tend to no longer produce quali- 
tatively new ideas. Only a new generation can do this relatively effortlessly and also 
produce great minds again. But even if there were no qualitative advances in knowledge 
in a generation, the natural process of generational succession would be mathematically 
more reliable than an insistence on massive life extension of Nobel laureates. 

Those following our reflections can resume with the self-purposefulness of the hu- 
man being. Ultimately, all human beings (like all beings) are interconnected and related 
to each other. This includes the fact that every human being has an intrinsic value that 
cannot be relativised, which can be politically described as human dignity. This individ- 
ual dignity is neither negotiable nor weighable but absolute (cf. Nida-Riimelin/Weiden- 
feld, 2018, p. 64ff). “The guarantee of dignity (...) is thus a central ethical guideline 
for the design of digitalisation” (cf. WBGU, 2019, p. 44[translated]). In humans, this is 
combined with subjective ethical insight that cannot be traced back to scientific laws 
or facts. Being a person, with ethical responsibility founded therein and the human 
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interconnectedness with all living beings always includes the emotionality and physi- 
cality of humans. People show respect for each other and appreciate each other in their 
materiality. (cf. Bauberger/Schmiedchen, 2019). 

Machines, on the other hand, cannot act autonomously, make decisions, or bear re- 
sponsibility. Their “autonomy” is always only derived. Therefore, the goals of all techni- 
cal products must be relativised to those values that are assigned to them by humans (cf. 
Bauberger/Schmiedchen, 2019). Even when considering compassion or consideration, 
we humans as sentient beings are fundamentally closer to all other living beings than to 
any machine we have created, no matter how intelligent it may be (ibid.). 

The end of human life in itself does not allow any external pressure for self-perfec- 
tion, as it contradicts human dignity. It is even more forbidden to give people future life 
value only in an improved form. Likewise, it must be categorically rejected if humanity is 
to be overcome in its biological evolution and replaced by an artificial life form created 
by him. 


5.6 Where Should the Journey Go and Why? 


In the last 250 years, humanity has succeeded in achieving tremendous improvements, 
some of them incredible, in almost all socio-economic areas: An increasing number of 
humans have not died of hunger, childhood diseases, appendicitis, or childbirth. All this 
has been achieved through human progress and social struggle. And it is important to 
make clear that scientific positivism, economic capitalism, and constant warfare since 
the mid 19th century have been the framework in which this has happened. 

However, it is equally important to point out that only the successful empower- 
ment of the poorer and weaker (e.g. with the help of trade unions, critical media, free 
and equal elections) have led to a socially accepted distribution of these technological 
achievements, at least in tendency on a national/regional level. In order to be able to 
discuss ethical questions of neurological self-improvements and trans/posthumanism 
even between different cultures, common starting points and questions are needed, on 
which people in all regions of the world can agree. To be able to answer these questions 
sensibly and in a way that is relevant to action, we in turn need answers that are not 
only suitable for Sunday speeches, but also survive in the cold realpolitik of interest 
representation. 

Possible starting questions could be: 


e Is life, as it is, good in principle, and is it sustained and justified by itself without 
any further intervention (e.g. merits, achievements, efforts)? 
e What do we humans want to be like in 2050, 2075, or 2121? 


A possible minimal consensus could be that the primary meaning and purpose of hu- 
man existence is to constantly renew life without causing too much damage; primarily 
through reproduction, secondarily through cultural achievements (carrying the torch 
forward). 
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An internationally more controversial, but probably necessary aspect, is that sci- 
entific findings since the beginning of the 20th century support the view of reality, in 
which everything that exists, at every point in space-time, is in constant connection and 
exchange with everything else that is. Consequently, the perfection ofthe individual is 
only a meaningful process if at least no harm is done to others as a result. However, this 
can only be claimed if possibly required scarce resources for this perfection process 
“are not lacking elsewhere” or would not make a “higher” contribution there to the 
“improvement” of humanity or the common good. However, the advocates of limitless 
human optimisation logically fail to provide this proof. So far, everything points to 
the fact that the said improvements are directly and indirectly at the expense of those 
people who (for whatever reason) are not able or willing to have these improvements 
made to themselves. Since this is not problematised by enhancement/trans/posthu- 
manism advocates, there is a well-founded initial suspicion that the possible potential 
for creating a worldwide two-class society is either not seen or condoned. While the 
rich are acquiring every opportunity to continuously upgrade themselves technically 
and prolong their lives in an unprecedented manner, the non-improved are becoming 
increasingly useless and disruptive. 

In economics, we know the principle of marginal utility and the principle of op- 
portunity cost. When using these two economic principles, we must consider what the 
desired improvements of and for the people will bring about and whether the trans-/ 
posthumanist path means more for humanity as a whole. A few simple considerations 
already point to serious ethical problems: improvements and life extensions lead c.p. to 
the individual being able to be active longer. This can be accompanied by a reduction 
in the birth rate or by a constant or even an increasing birth rate. In the first case, 
humanity would become older through enhancement and life extension. In the other 
two cases, more people would live at the same time than would be the case with normal 
mortality. As shown above, there is always a serious risk that humanity will slowly freeze 
socioculturally as the proportion of younger people who keep giving birth to new ideas 
decreases relatively. In the latter case, the Earth would reach the objective resource 
limits even faster. 

Technological posthumanism triggers much more serious problems: 


e An artificial intelligence that reaches the stage of a singularity can, following its 
basic programming, be “benevolent” to humans and “behave” in the interest of 
humanity. In this (logically not very probable) case, we would have created a 
paternalistic “God” who protects and develops human life. 

e But the singularity could also be equally “benevolent” towards humanity and the 
well-being of the planet, and “realise” that humans usually do not behave for their 
own good. Then this paternalistic “God” would take us by the hand and lead us 
into a future, which he deems as correct. 

e Or, the singularity feels more “obliged” to the well-being of planetary life as a 
whole. In that case, people would have to accept numerous restrictions, would 
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possibly be decimated, or, in extreme cases, wiped out because they have (so far) 
behaved in a harmful manner without understanding. 


Each of the just-outlined alternatives shows that the benefit-danger ratio cannot really 
be considered to be beneficial for humanity. Accordingly, the final question remains to 
be asked: Who would benefit from a trans-/posthumanist future? 
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Legal Frameworks and Standards 
of Digitalisation 


Introduction 
Jasmin S.A. Link 


Some years ago, I have asked a peace researcher and member of the Federation of 
German Scientists (VDW): If you had known that nuclear bombs could be developed, 
would you have sought to discover nuclear power nonetheless? His answer was: Yes. 
Because that way global peace could be guaranteed for a long time via international 
treaties. It could be expected that everything that can be done, will be done. Thus, if he 
had not been the one to make a certain discovery, somebody else would have been the 
one. But the peace researchers’ achievements have demonstrated that it is possible to use 
a new technology for the international good such as international stability and peace, 
which would be assured by bilateral or multilateral treaties.” 

Just as important is the VDW activity with regard to the technological impact 
assessment of the digital transformation. Early requests to define the necessary legal 
framework of digitalisation, digital IT networks, and research and development and 
implementation of so-called artificial intelligence systems (AI) can have crucial impacts 
on society. Part I of this compendium has shown that digitalisation, digital networks, 
and AI development can cause disruptive changes and can pose fundamental challenges 
for technical, democratic, functional, economic, human health, and social development 
security. The digital developments can even trigger initiatives that question the right of 
human existence. Potentially, within many societies the ongoing digital transformation 
already seems “without alternatives”. 

In part II, the legal aspects of the digital transformation are assessed. Who is to be 
made responsible for potential “collateral damage’, e.g. of autonomous cars? Can ma- 
chines be made responsible? If so, do they have “rights” in exchange, too? Accordingly, 
would human rights need to be applied to machines as well? Who is right in contradic- 
tory testimonies: humans or machines? 

Which social standards are already developed to use existing political structures for 
the design of the process of digitalisation? Which additional legal measures are planned? 
Can and should there still be a copyright for intellectual property? Which and how? 

And how can lethal autonomous weapon systems be designed to fulfill existing mar- 
tial law acts to avoid civil victims? How does the development or existence of autono- 
mous weapons change not only warfare but also international diplomacy? 

The following chapters of part II will discuss these and further questions in detail. 

In chapter 6, Stefan Bauberger contrasts the transhumanistic perspective, which was 
presented in chapter 5, with a machine-ethical perspective. He scrutinises the ethical 


33 This does not mean that all peace researchers at VDW share this opinion. It is ethically problematic to 
just do everything that is possible because otherwise someone else would do it instead. This requires 
further scientific debate with regard to the responsibilities of science in general. 
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aspects of so-called autonomous actions of a machine that is controlled by AI. Particu- 
larly, when checking whether the concept of human dignity can be applied to machines 
he discovers ethical problems. Even the question whether a machine controlled by AI 
can develop consciousness is discussed briefly. 

In chapter 7, Christoph Spennemann assesses from a legal perspective who can be 
made accountable for potential malfunctions of AI controlled systems, robots, or au- 
tonomous cars. Existing laws are evaluated with regard to their applicability in the AI 
context and it is analysed whether they can be extended to include the digital transfor- 
mation or whether the EU needs to design new legal frameworks to assure the insur- 
ability of AI products on markets. 

The subject of standards in the digital world will be addressed in chapter 8 in two 
parts. In part A, Eberhard K. Seifert describes classical standardisation processes and 
organisations. It is explained how national and international mechanisms interact. Ap- 
plied to the example of digitalisation, it is assessed which goals can be addressed and 
how some mechanisms of action can potentially create undesirable side effects. For ex- 
ample, the bottom-up self-organisation can result in an economic politically motivated 
top-down effect on EU member states. 

Additionally, in chapter 8 B, Michael Barth analyses the international standardisa- 
tion structures focusing on digitalisation, and he points out how powerful the politi- 
co-economic effect of standardisation processes can be, especially in an international 
dynamic. He describes which organisations and nations are currently taking which 
international positions in the context of the digital transformation, digital networks, 
and AI. He emphasises why this positioning is of increasing fundamental concern for 
Germany and the EU. 

In chapter 9, Christoph Spennemann analyses the legal dimensions of which kind 
and how intellectual property is protectable in the context of the digital transformation, 
digital networks, and especially the application of AI. He is aware of internationally 
diverse legal frameworks shaping the competition in innovation in the development of 
AI systems, and also in connection with the potential development of strong AI. The 
trade-off between innovation capacity and property rights may become necessary, just 
as the founding of a committee of external experts, as well as the analysis of potential 
indirect international future effects of regional legislation. 

In chapter 10, Götz Neuneck critically assesses the current development of drones 
and autonomous arms systems from a technological and peace research perspective, 
particularly with regard to possible consequences for warfare, military operations, and 
impacts on civilians. Even though already existing arms treaties might be interpreted in 
a particular way or may be extended to include autonomous weapons, humans still have 
a long way to go to internationally regulate such arms systems. Therefore, it is necessary 
to first create awareness of the real relevance of potential scenarios of armed conflict that 
may result from the possible development or malfunction of AI. 

When reading these chapters, please note that the perspectives presented originate 
from different scientific disciplines. Consequently, the exact meanings of the used terms 
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such as robots, autonomous machines, or artificial intelligence are based on the scien- 
tific literature of the particular discipline. Therefore, the specific use of the terminology 
may vary from the technical definitions that were presented in chapter 2 by a computer 
scientist. 
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Machine Rights 


Stefan Bauberger * 


There are various calls in the literature for a personality status for robots or correspond- 

ing rights. The European Parliament’s call for this in 2017 is particularly significant: 
“The European Parliament (...) 59. Calls on the Commission, when carrying out an impact assessment 
of its future legislative instrument, to explore, analyse and consider the implications of all possible 
legal solutions, such as: (...) f) creating a specific legal status for robots in the long run, so that at 
least the most sophisticated autonomous robots could be established as having the status of electronic 
persons responsible for making good any damage they may cause, and possibly applying electronic 
personality to cases where robots make autonomous decisions or otherwise interact with third parties 
independently” (European Parliament, 2017). 


This demand is related to other proposals for regulating claims for damages in the event 
of damage caused by robots.*° The demanded status of the “electronic person” is thus 
seen as analogous to the status of the legal person of partnerships or to the Anglo-Saxon 
trust concept. However, the justification (“where robots make autonomous decisions”) 
also hints at transhumanist arguments and thus goes beyond the context of claims for 
damages. The only demand was to examine whether such a regulation makes sense. 

As shown below in chapter nine by Christoph Spennemann on intellectual property 
rights, there is another argument to consider regarding the issue of robot rights, namely 
the unresolved attribution of copyright of AI products. 

An open letter from more than 150 “political leaders, Al/robotics researchers and 
industry leaders, physical and mental health specialists, law and ethics experts” articu- 
lated widespread opposition to this demand from the European Parliament (Nevejans 
et. al., 2018). The European Commission did not follow the call. 

The open letter rejects the claim to give robots a legal status corresponding to a 
natural person, but also the claim to recognise robots as legal persons. Both demands 
can be discussed separately. 


6.1 Artificial Intelligence as a Legal Entity? 


Shawn Bayern as well as several European authors have pointed out that there are al- 
ready loopholes in US and possibly also in European laws, through which an AI (which 
does not necessarily have to be connected to a robot) can obtain the status of a legal 
entity. This makes clarification of this issue all the more important (see Bayern, 2015 


34 Cf. with another chapter by the author: Bauberger, Stefan: Welche KI? Munich: Hanser-Verlag 2020. 
And a submitted contribution to the planned Handbuch KI. 
35 Cf. Chapter 9 of this book, by Christoph Spennemann. 
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and Bayern et.al., 2017 and Burri, 2018). Kaplan also conceives ofthe scenario ofan Al 
(possibly a robot) that buys itself, in analogy to how, at the end of slavery in the USA, 
slaves ransomed themselves (cf. Kaplan, 2017, p. 123). 

The construction of the “legal person” was created for business purposes so that not 
all actions of a company or association or other entities have to be attributed to the 
respective natural persons behind them. A legal person ensures a stability of contractual 
relationships beyond the respective natural persons and prevents the need for compli- 
cated division of legal obligations (e.g. in the case ofa public corporation among the many 
shareholders). Furthermore, it enables a limitation of the liability of the respective natural 
persons. The goals of the legal person, generally economic goals, are given to it by the 
natural persons, but are then assigned to the respective legal person. Every legal person 
needs - according to the previous concept - natural persons as decision-making bodies. 

This last point is the starting point for the idea that Als could act as electronic 
persons along the lines of the already established legal persons, if they can make “au- 
tonomous decisions” (formulation of the European Parliament, cf. above), i.e. if they 
no longer need human decision-makers. The main purpose of the electronic person 
would then be an appropriate regulation of liability claims in case of damages whose 
causation cannot be attributed to a natural person because the decisions in question do 
not originate from a natural person, or the attribution of copyright or patent rights to 
AI products. 

In order to assess this concept, it must be clarified, on the one hand, what “auton- 
omous action” by machines means and, on the other hand, whether such a regulation 
brings societal benefits. For this purpose, the following argument is limited to the area 
of liability claims. 

A certain form of autonomy can be attributed to all self-moving machines. The sig- 
nificant difference of AI (AI in the sense of machine learning) is that it acts inherently 
unpredictably. The purpose of machine learning is to deploy the corresponding systems 
in high-dimensional state spaces that cannot, in principle, be fully anticipated. Machine 
learning then means either that the systems behave analogously but in a generalised 
manner to given training data (pattern recognition), or that they adapt their behaviour 
through “self-learning”, i.e. on the basis of given rules and optimisation parameters. 

As shown in chapter nine, when AI is used in areas with a high potential for dam- 
age, such as motor vehicle liability insurance, an (at least largely) unlimited claim for 
compensation by the injured parties is socially desirable, without the injured parties 
having to prove intentional misconduct on the part of the manufacturers or the users. 

In addition to the question of liability insurance, it must also be clarified whether 
and to what extent product liability is possible for AI products (cf. Chapter 9). There 
is the argument that product liability for AI is anti-innovation, especially because the 
risks are not always foreseeable. In this regard, however, it should be noted that although 
manufacturers cannot assess these risks completely, they are in the best position of all 
actors to do that. They are the only ones with access to the design principles and training 
data for an AI. Both will generally be trade secrets. Therefore, from a macroeconomic 
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perspective a good balance between the risks and opportunities of Al can only be en- 
abled by extensive product liability. 

It follows from these considerations that a separate legal status for the legal entity 
could bring advantages to manufacturers, but that these advantages would be associated 
with considerable disadvantages for society in terms of liability regulations. This status 
has the potential of socialising the risks, which no one can assess, in favour of the pro- 
ducers. It should therefore be ethically rejected from this perspective. 


6.2 Al as a Natural Person? 


It has already been mentioned above that the text of the European Parliament can also 
be read as being based on transhumanist arguments, as presented in chapter five. This 
means that because machines are capable of “autonomous decisions” and “independent 
interaction with third parties’, no difference in principle can be found between humans 
and machines. In addition, there is the concept in the corresponding literature that 
advanced Als can be programmed to behave ethically so that they take responsibility 
for their actions. 

The rejection of such concepts is sometimes criticised as anthropocentric. The point 
of comparison invoked is the often-repeated struggle for equality throughout history, 
e.g. in the course of the abolition of slavery and the civil rights movement in the USA, 
through which personal rights and civil rights were extended further and further. In 
retrospect, the previous restriction of personal rights is clearly identifiable as unjust. 
A correspondingly backward and unjust position is assumed by advocates of personal 
rights for robots by those who criticise these rights. The robot Sophia was granted 
citizenship in Saudi Arabia in 2017, but given Sophia’s real capabilities, this can only be 
understood as a symbolic marketing action. 

The radical transhumanist view of AI (cf. e.g. Moravic, 1999; Kurzweil, 2015) even 
assumes that robots will overtake humans in development and thus replace them as 
the highest stage of evolution. When development has “progressed” to this point, it is 
ethically imperative that humanity relinquishes its claim to leadership and hands it over 
to AI (cf. Chapter 5). 

Such considerations build on a reductionist view ofthe world and human beings. 
Within this view, the concept of a “higher stage of development” cannot be defined as an 
ethical category, but only on the basis of the factual fact of who prevails in development 
against other forms of life or existence. Ethically motivated demands of robot rights are 
therefore self-contradictory if they are based on the postulated higher development. 

Two arguments still need to be considered: The autonomy of Als and the question 
of whether Als can develop consciousness. 

“Autonomy” of AI means - as analysed above - practical unpredictability of action 
due to great complexity. If autonomy is understood in this way and put forward as an 
argument for continuity between human action and the “action” of robots, then this 
presupposes that the autonomy of action of humans is reduced to the unpredictability of 
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this action (due to the complexity ofthe organism, especially the brain). This reduction 
contradicts the foundations of ethical action, which Kant defines as follows: “Autonomy 
of the will is the constitution of the will, by which the same is a law to itself (independent 
of all constitution of the objects of the will)” (Kant, 2012). This conception of autonomy 
also underlies all current legal systems, which understand the free action of man in 
such a way that new causes are set in it. Only then can these actions be assigned to the 
respective autonomous subject who bears responsibility for them. 

In this respect, a justification of personal rights for Als based on this empirical con- 
cept of autonomy dissolves the foundations of ethics and jurisprudence in their present 
form and must be rejected from an ethical perspective. To base an ethical claim to 
personal rights for robots on this is again self-contradictory. 

A stronger argument for a person status of Als is not based on this empirical notion 
of autonomy, but on a claimed ontological continuity between humans and Als. In par- 
ticular, it is pointed out that Als could develop consciousness. However, this argument 
suffers from the problem that the existence of consciousness cannot be operationalised. 
Consciousness is only accessible in the perspective of the 1st person, the ego, not in 
an objective external view (Nagel, 1974). The concrete argument that AI can develop 
consciousness is based on the reductionist thesis that consciousness, including human 
consciousness, is a result or concomitant of complex information processing. This par- 
adigm is widespread in the philosophy of mind. However, the concept of information 
cannot be justified in naturalistic terms (Janich, 2006), which means that this paradigm 
is based on an inversion of the logical reasoning. Furthermore, it is highly questionable 
to link consciousness to information processing. Rather, consciousness precedes discur- 
sive thinking and is thus not dependent on it. 


6.3 Human Dignity 


Ultimately, a demand for rights for robots or for Als in general (corresponding to a per- 
son status) must be based on granting them a dignity corresponding to human dignity. 

However, as Frank Schmiedchen already shows above, it is highly questionable and 
does not correspond to the established legal view to attach human dignity to intelligence. 
Mentally handicapped people would otherwise have less dignity and, in general, the 
dignity ofpeople would be graded according to their intelligence. But that would be the 
logical consequence if robot rights were claimed on the basis of intelligent behaviour 
of robots. 

Human dignity is based on the end in itself of human beings. In Kant’s formulation, 
this reads as follows: “Act in such a way that you use humanity, both in your person and 
in the person of everyone else, at all times as an end, never merely as a means” (Kant, 
2004, p. 79). This self-purposefulness also applies to other living beings in a graduated 
manner (or, according to certain bioethical views, in the same manner). It is based 
on the fact that the goals of their actions are inherent in living beings. In view of the 
development of robots and Als that imitate human action, it is important, in contrast 
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to the self-purposefulness of living beings, to strengthen the techno-ethical principle 
that machines do not have a purpose in themselves, that rather their meaningfulness 
depends on the goals that are given to them by humans, which I have already explained 
above. The autonomy of Als in this context means - as shown above - only a pragmatic 
unpredictability of the behaviour of these machines, and it has as an ethical consequence 
to demand high safety standards for their development and application. 


6.4 Outlook 


It is to be expected that the discussion on a legal status for AI, which is just beginning, 
will gain in importance with the spread of applications of AI. 

Firstly, the economic interests outlined above will grow to escape product liability 
with such a construct, especially for devices whose behaviour is not completely pre- 
dictable. The companies involved will claim a public interest in not falling behind their 
competitors in international competition in this innovative field. 

Secondly, psychological research shows that people tend to project that everyday ob- 
jects in their everyday world, and especially robots, are alive (cf. Darling, 2016 and other 
articles by this author). This natural human tendency can combine with the concept of 
machines as natural persons with philosophical speculations of a completely different 
content, which question any special position of humans from a naturalistic position. 

Against these tendencies, two principles should be strengthened at an early stage. 
The first of these principles is that technology should not be an end in itself. Technology 
is created by humans to make life easier and must always be directed towards this end 
(cf. Chapter 4). 

The second principle is human dignity and in particular the aspect that this dignity 
does not depend on the intelligence of the human being. This principle arises from 
the rootedness of this human dignity in the autonomy of the living. It leads, properly 
understood, in the debate on the status of AI, to humans rediscovering their kinship 
with other living beings, especially sentient living beings, as opposed to their supposed 
kinship with machines, however intelligent they may be. 
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Liability Issues 
Christoph Spennemann 


A variety ofliability issues can become relevant in the context of digitalisation. The chap- 
ter therefore builds on the philosophical foundations of the previous one. It focuses on 
liability issues in the specific context of Aland is based on German law. Al can violate tan- 
gible (life, health, property) as well as intangible legal assets (intellectual property rights). 


7.1 Al and Liability for Material Damage 


It is much-discussed case studies, which have already been mentioned in several places 
in the book or will still appear, that are relevant here: A self-driving car controlled by 
AI injures a passer-by against the will of the driver. A surgery performed with the help 
of a robot leads to an injury to the patient against the will of the doctor performing the 
surgery. In both cases, the AI users followed the instructions given to them. The AI was 
faulty, which was not recognisable to the user. Here the question arises as to who is liable 
for the damage that occurred. The manufacturer of the product containing the defective 
AT? The programmer of the software on which the AI is based? The AI user? In any case, 
AI is not liable itself due to the lack of its own legal personality.°® 


7.1.1 Liability of the Manufacturer or the Programmer 


The basis for a liability claim by the injured party is usually non-contractual, as there is 
no contractual relationship whatsoever between the manufacturer and the victim of the 
defective AI, as in the aforementioned examples of the pedestrian injured by an intelli- 
gent vehicle and the patient injured by a surgical robot. Therefore, contract-independent 
claims under the Product Liability Act (ProdHaftG) and the German Civil Code (BGB) 
come into consideration. The ProdHaftG has introduced a so-called “strict liability” of 
the manufacturer, which arises even without fault only due to placing a danger on the 
market.” It is true that the prevailing opinion in research and practice assumes that Al is 
not to be regarded as a product within the meaning of the ProdHaftG, as this presupposes 
a movable object and it is doubtful whether software and algorithms in their non-physical 
nature fulfil this prerequisite (Federal Ministry for Economic Affairs and Energy [BMWi], 


36 See below, Chapter 9 on Intellectual Property Rights. For more details on the problem of a separate 
legal personality for AI, see Chapter 6, Machine Rights. 

37 § 1 (1) ProdHaftG. Liability extends to damage to life, physical integrity, health and, with certain 
restrictions, property. 
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2019, p. 16). This means that the programmer is not liable under the ProdHaftG. In 
combination with a physical object such as a motor vehicle or a surgical robot, however, 
such a product exists in any case and the manufacturer of the motor vehicle or the robot 
that installed the AI can be considered liable. However, product liability presupposes the 
existence of a defect in the product, and this defect must be proven by the injured party.*® 
This is likely to prove very difficult for an outsider in the field of AI, where not even the 
manufacturer may have a comprehensive understanding of the inner workings of the AI. 

If the injured party succeeds in establishing the difficult proof, the product manu- 
facturer can in turn try to exclude his liability by invoking various grounds for excul- 
pation.” In the field of AI, the ground mentioned in § 1 (2) No. 2 ProdHaftG could be 
particularly relevant, according to which liability is excluded if it can be assumed that 
the product was not yet defective when the manufacturer placed it on the market. AI can 
change in unforeseen ways in the course of its operating time and achieve unplanned re- 
sults or cause damage. Although it has been pointed out that in such cases the fault does 
not lie in the knowledge trained into the AI, but in the programming that was faulty from 
the outset, which made undesirable developments possible in the first place (BMWi, 
2019, p. 17). However, cases are conceivable, in which the original programming cannot 
be considered faulty. Accordingly, a manufacturer can be exculpated ifhe can prove that 
the defect could not have been detected according to the state of the art in science and 
technology at the time the manufacturer placed the product on the market.*° 

An alternative to product liability is recourse to “producer liability” under the German 
Civil Code." It is true that the producer is only liable here in the case of fault (intent or neg- 
ligence). However, in the case of producer’s liability in particular, such fault is presumed 
due to a reversal of the burden of proof. If the producer is unable to rebut this presump- 
tion, he is liable for the defective AI. However, this presumption only comes into effect 
after the injured party has first proven the defect. In this respect, producer liability presents 
the injured party with the same difficulty as product liability, namely, proving an error 
in an algorithm, the exact functioning of which may also be unclear to the manufacturer. 


7.1.2 User Liability 


Here, too, the basis for a claim is the tort law of the BGB. However, in contrast to the spe- 
cial producer liability (see above), the conventional rules of the burden of proof apply 
to the AI user. The injured party must therefore prove that the user acted intentionally 
or negligently in handling the AI.*? If the user has adhered to the operating instructions 


38 § 1 (4) ProdHaftG. 

39 Cf. § 1 (2) and (3) ProdHaftG. 

40 § 1 (2) No.5 ProdHaftG. 

41 This is an instrument developed by case law on the basis of § 823 (1) BGB (law of torts), the exact 
form of which is not expressly formulated in law. 

42 The driver of the vehicle is only liable to pay damages in case of fault (intent or negligence), cf. § 18 
(1) Road Traffic Act (StVG). 
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of the AI, such proof of fault will not be possible. User liability can therefore be virtually 
ruled out under German law. 


7.1.3 Special Case: Liability of the Vehicle Owner 


In the Road Traffic Act (StVG), the legislator has established a strict liability of the 
vehicle owner (n.b. not the vehicle driver*?), which applies in the case of self-propelled 
vehicles. Liability is linked to the existing liability insurance of all vehicle owners. As 
with the ProdHaftG, a motor vehicle owner is liable solely for the operation of a source 
of danger, regardless of his or her own fault. Unlike under the ProdHaftG, however, the 
injured party does not need to prove a fault in the AI. The keeper is liable solely because 
of his capacity as keeper. In the case of self-driving vehicles, injured parties therefore 
have the best prospects for compensation. 


7.1.4 Contractual Recourse Claims 


In the case of liability of the manufacturer or the vehicle owner, the liable party (or the 
insurance company that pays for the damage) will have an interest in being reimbursed 
for the damage incurred. Contractual claims for damages against the seller of the prod- 
uct containing the defective AI (e.g. a car or a surgical robot) may be considered. The 
latter will in turn address its own contractual partners upstream in the manufacturing 
chain, e.g. a car supplier, in order to obtain compensation for damages due to defective 
performance. At the end of the chain of recourse is most likely the AI programmer. How- 
ever, a problem arises here that is comparable to the enforcement of claims from product 
liability: each injured party must prove the existence of a defect to the respective con- 
tractual partner ahead of him in the production chain. If the AI itself was programmed 
incorrectly, such proof could be very difficult. And even if the error in the AI can be 
conclusively shown and the claims for damages finally reach the software programmer, 
another difficulty arises here: programmers in many cases use so-called open source 
software (OSS), the various elements of which (and thus also incorrect programming) 
cannot be clearly assigned to a specific programmer. Accordingly, contractual liability of 
individual developers is excluded in the terms of use of OSS (BMWi, 2019, p. 17). The 
innovation advantages of the division of labour in OSS thus prove to be a disadvantage 
for contractual claims for damages. 


Summary of Legal Status Quo and Existing Proposals for Reform 

The prospects of a person injured by AI to obtain compensation are not always equally 
favourable. Very good prospects exist in the special area of road traffic, as the vehicle 
owner is subject to strict liability. Less promising are claims for damages against the 


43 § 7 StVG. 
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manufacturer of Al-operated devices outside road traffic. Both the producer liability of 
the BGB and the product liability ofthe ProdHaftG require the injured party to prove a 
defect. Such proof can certainly succeed in the case of analogue products whose mode 
of operation is clearly transparent. In the case of misguided algorithms, on the other 
hand, this proof seems unreasonably difficult. The possibilities for exculpation of the 
manufacturer provided for in the ProdHaftG (in particular the proof that the product 
is free of defects before it is placed on the market) create additional legal uncertainty 
for injured parties. 

Claims against users ofthe AI, such as a motor vehicle driver or a surgeon, appear 
even more difficult, as it will hardly be possible to prove fault on their part. This also 
applies to possible tort claims against the programmer ofthe AI ($ 823 BGB). Claims 
for damages against the programmer based on product liability are also ruled out, since 
software is not a product in the sense ofthe ProdHaftG according to today’s predomi- 
nant understanding. 

Both the European Commission and the Federal Ministry of Economics and Tech- 
nology (BMWi) have commented on questions of liability for defective AI. The BMWi 
considers such cases to be covered in principle by the ProdHaftG and producer liability 
and therefore sees no need for action by the legislator as long as there is no independent 
AI (BMWi, 2019, p. 19)**. However, in its analysis, the BM Wi does not address the prac- 
tical difficulties with the burden of proof that affect the injured party. This problem is 
discussed by the EU Commission. In a report from February 2020, it provides compre- 
hensive considerations on the applicability of existing liability regimes to Al-powered 
technologies (European Commission, 2020). In this context, the Commission does not 
yet make any recommendations but indicates, which approaches it would like to pursue 
further and secure by seeking opinions. In doing so, it draws on an expert report from 
2019 (European Commission, 2019). The following approaches of the Commission will 
be briefly discussed here: 


e In order to ease the burden of proof problems described above for the injured 
parties of Al-operated technologies, the Commission is considering strict lia- 
bility for operators of AI technologies, comparable in result to motor vehicle 
owner liability. Both liability of the direct operator and of persons who exercise 
permanent control over the function of the technology, for example through on- 
going updates of the operator’s software, are possible. The person who exercises 
the strongest control over the technology, either directly or indirectly, should 
be liable (European Commission, 2019). However, the Commission advocates 
strict liability only for those products and services that are used in public areas 
and may expose the public to significant risks to life, health, and property. An 


44 However, the BMWi advocates an expansion of the definition of error in the ProdHaftG. So far, this 
only covers defective programming of algorithms, but not the training of an algorithm with erroneous 
learning data (ibid., p. 18). 
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extension of strict liability to all technologies operated by Al is rejected. In this 
context, the Commission expresses the fear that too far-reaching strict liability 
could delay the introduction of new AI technologies (European Commission, 
2020)*°. The 2019 expert report also doubts whether insurance companies would 
be prepared to comprehensively cover the risks caused by AI technologies if the 
quantification of a loss and thus the sum insured should prove too complex due to 
a lack of experience with new technologies (European Commission, 2019, p. 61). 
e For the operation of all other AI applications, which in the Commission's view 

“are likely to constitute the vast majority [...] the Commission is considering 

adapting the rules of evidence of existing liability regimes to the specificities of 

new technologies: 

o In the case of product liability, the manufacturer of AI technology could, in 
contrast to the currently valid ProdHaftG, also be liable for such defects of 
the product that only materialised after the product was placed on the mar- 
ket, for example through subsequent updating software of the AI over which 
the manufacturer exercised control, or in the case of subsequent changes to 
the AI originally placed on the market due to the self-learning properties 
programmed before the product was placed on the market. Once the injured 
party has proven damage caused to him by the digital technology, also in 
derogation from the current ProdHaftG, the fault is presumed to be at the 
manufacturer’s expense if it was unreasonable for the injured party to prove 
that the manufacturer failed to comply with certain safety standards due to 
associated costs or disproportionate practical difficulties (European Com- 
mission, 2019, p. 42). The same applies if the manufacturer fails to comply 
with an obligation to provide documentation that can reveal errors in the AI 
software (ibid., p. 47). Conversely, the Commission considers contributory 
negligence on the part of the injured party if the latter has failed to update 
the software that has damaged him in a reasonable manner (European Com- 
mission, 2020, p. 18). 

o Under fault-based liability, operators should be obliged to select the appropri- 
ate AI system, monitor it and maintain it. Manufacturers should be required 
to design, describe and market AI systems in such a way that an operator 
can comply with the above obligations. In addition, manufacturers should 
adequately monitor AI technology after it has been placed on the market. 
(European Commission, 2020, p. 44). The concrete definition of these obliga- 
tions would determine the extent of the burden of proof on the injured party. 
If he meets this burden, fault on the part of the operator or manufacturer 
would be presumed. From the point of view of German liability law, this 
would not be an innovation with regard to the manufacturer. As described 


45 With reference to European Commission, 2019, and the analysis found there on p. 61. 
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above, the ProdHaftG waives fault. The producer liability drafted by case law 
has already established a presumption of fault. With regard to user liability, 
it is also questionable whether the Commission’s proposal would sustainably 
facilitate the evidence situation of the injured party. After all, the injured 
party would have to prove the AI operator’s defective selection, monitoring, 
or maintenance of an AI system. Here, the above-mentioned approach of 
strict strict liability for technologies used in the public sector might prove to 
be much more advantageous for the injured party. 

e In addition to easing the burden of proof, the Commission is considering other 
important changes, such as in particular an expansion of the concept of product 
in the EU Product Liability Directive. As already stated, the prevailing opinion 
in Germany is that software is not covered by the ProdHaftG. Only if the concept 
of product is explicitly extended to software, software programmers could also 
become liable to recourse, for example against a liable manufacturer who has in- 
corporated defective software into a product that has injured the injured party.*® 


7.1.5 Comment / Recommendations for Action 


Assuming that in the future AI will not only enable autonomous driving but also influ- 
ence other areas of technology, it seems desirable to ensure uniform liability for material 
damage. Why should obtaining compensation for a failed surgery be more complicated 
than in the case of a car accident, even though the legal interests of the injured are the 
same? It is therefore obvious to introduce strict liability construed as the liability of 
motor vehicle owners also in areas outside road traffic. The injured party should always 
be able to claim this against the person who directly injured him. However, this would 
require the introduction of a general insurance obligation for Al-related damage, with- 
out which doctors or hospitals, for example, would hardly engage in AI-based treatment. 

As there are hardly any liability cases due to faulty AI at this point in time, it cannot 
be clearly assessed whether the Commission's concerns regarding strict liability affecting 
all AI applications are justified. The example of motor vehicle owner liability shows that 
strict liability with simultaneous insurance obligation need not be an obstacle to the 
introduction of new technologies. Even the operation of a motor vehicle can result in 
very high losses for the insurer, without this being detrimental to the development of 
the car industry. However, we have to cede to the Commission that at this stage it is also 
unclear how well insurers will be able to determine the risk caused by largely unknown 
AI technologies. It would probably make sense to introduce strict liability in some areas 
first, as proposed by the Commission. 

In the interest of effective consumer protection, these areas should cover cases where 
the user is potentially exposed to a high degree of harm to life, health or property, such 


46 Cf. § 5 ProdHaftG in conjunction with § 426 (2) BGB. 
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as in the examples of self-driving cars mentioned earlier, but also in the use of robots in 
surgery and in nursing and care of the elderly. The liability requirement mentioned by 
the Commission for the use of Al-powered products and services in public areas must 
not lead to an exclusion of liability for use by private carriers or private individuals (e.g. 
in the use of care robots in private households). Rather, strict liability should kick in 
when someone operates or controls a technology that is available for use by the public. 

With regard to other AI applications, i.e. if in particular there is no serious risk to 
life, health, or property, European product liability law should be amended in accor- 
dance with the Commission’s proposals. A reversal of the burden of proof regarding the 
existence of a product defect in favour of the injured party seems particularly important 
here (see above). This would make it much easier for the injured party to sue for dam- 
ages. However, it also seems fair for the manufacturer to prove contributory negligence 
on the part of the user if the consumer has failed to update the software in a reasonable 
manner. 

However, one should be open to further adjustments of liability law in the sense of 
extending strict liability to all areas of technology if it becomes apparent that compul- 
sory insurance for the operation of selected technologies works, i.e. does not slow down 
technological innovation and is also well accepted by the insurance industry. 

The EU-wide introduction of consumer-friendly liability rules could, because of the 
size of the European market, lead to insurance companies also offering insurance for 
the operation of AI-based technologies outside the EU. 


7.2 Al and Liability for Immaterial Damage 


If intellectual property rights are infringed by AI - e.g. by unauthorised copying of 
copyright-protected works in the context of an Al-driven mass data analysis - the stakes 
are somewhat different than in the case of material damage. Unlike in the case of failed 
operations or a misguided motor vehicle, the AI functions perfectly, but for that very 
reason infringes certain intellectual property rights. The rights holder can therefore 
only address the person who uses the error-free Al in a way that infringes rights, e.g. a 
researcher who analyses masses of data for commercial purposes and infringes copy- 
rights in the process. 

In this context, there are civil law claims to cease and desist from further infringe- 
ments and, in the case of culpable (i.e. intentional or negligent) infringement, claims 
for compensation for the damage caused by the infringement.*” Culpable infringements 
of copyright and trademark rights are also criminal offences and can be punished with 
imprisonment.*® 


47 For the field of patent law, see Section 139 (1) and (2), Patent Act. For the area of copyright law, see 
Section 97 (1) and (2), Copyright Act. 

48 See Sections 106 et seq. of the Copyright Act; Sections 143 et seq. of the Trade Marks and Other 
Distinctive Signs Act. 
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Depending on the individual case, the degree of independence of the AI can also 
cause ambiguity here: the user of the AI could object that he or she can no longer un- 
derstand the internal processes of the Aland therefore acted without fault. However, this 
would have no effect on claims for injunctive relief that do not require fault. Moreover, 
the credibility of such an objection is doubtful in many cases, as the user is usually 
likely to be aware of the tasks the AI can take over. This usually justifies the assumption 
of at least negligent infringement. Thus, the applicable liability regime for immaterial 
damages does not currently require any special adaptation to new AI technologies. 


7.3 Concluding Observations 


Well-functioning liability rules are an essential component of future digitalisation. They 
provide the legal certainty without which manufacturers and users would hesitate to 
invest in or use Al-powered products. Since internal technical processes and function- 
alities of algorithms and software are generally more difficult to understand than in the 
analogue sector, existing rules must be changed to enable AI users to be compensated 
for the damages they have incurred. The EU Commission's proposals for an Al-oriented 
revision of EU product liability law are a step in the right direction. The EU should move 
forward swiftly to adapt its legal framework. Coordination with other global actors in 
international forums should be sought, but should not influence the pace of EU legal 
changes. Rather, as in the case of the General Data Protection Regulation, EU standards 
can be expected to influence corporate behaviour far beyond the EU thanks to the size 
of the Single Market. For example, the successful introduction of liability insurance for 
producers and users of Al-based products and services could encourage insurance com- 
panies to offer similar business models in other parts of the world. By moving forward 
decisively, the EU can set new global standards here. 
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A. Norms and Standards for Digitalisation 
Eberhard K. Seifert 


Introduction 


Regulations based on consensus standards are of crucial importance for the further de- 
velopment of digitalisation, networking and artificial intelligence (AI): because whoever 
sets the standards determines the market! The economic benefit of standardisation is 
estimated at around 17 billion Euros annually in an updated study for DIN on the first 
decade of this millennium (cf. Blind, K. et.al., o. J.). 

At the beginning of 2018, the German Institute for Standardisation (DIN) pro- 
claimed in a four-page position paper that digitalisation only succeeds with norms and 
standards - successful digital transformation through active standardisation: 

“1. norms and standards are the first means of choice to achieve technology transfer and combine it 

with global market penetration. 

2. cross-industry and inter-disciplinary cooperation platforms, e.g. the German ‘Industrie 4.0’ plat- 

form, are particularly suitable for expanding Germany’s competitive positions if sufficient attention is 

paid to standardisation. Policymakers should therefore support the development and use of voluntary 
standards. 

3. Start-ups and Small and Medium Enterprises (SME) should be encouraged to utilise and support 


standardisation via government funding initiatives. This will make market access easier and also raise 
the existing innovation potential in terms of technology transfer” (DIN, 2018, Preamble). 


Contrary to their actual enormous economic and, in some cases, societal significance, 
standardisation and the processes by which it comes about are hardly known and 
discussed - not even in research and teaching.’® Responsible institutions and actors, 
governance structures, and the processes of national and international negotiations for 
generating, editing, or further developing standards are largely unknown.°° 

Section 8.1 of Part A of this chapter therefore first provides a brief insight into the 
function, structure and institutional landscape of the usual privately organised standard- 
isation processes at national, European and international level. Based on this, Section 
8.2 provides insights into (inter)national standardisation activities on Industry 4.0 and 
artificial intelligence. Due to the complexity and technical nature of the subject matter, 
only the essentials can be addressed here, but this overview is extremely important for 


49 An exception is the European network EURAS initiated/registered in Germany (https://www.euras. 
org/). 

50 See, for example, the instructive anthologies by Hawkins, et.al. (Eds.) (1995) and by Hesser, et al. 
(Eds.) (2006). 
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the purpose ofthe book. Part A ofthe chapter concludes in 8.3 with references to possi- 
ble restructurings of standardisation processes that have been common up to now. This 
is followed in Part B by an assessment of the growing geostrategic significance of norms 
and standards outside the standardisation organisations by Michael Barth.” 


8.1 Insights Into the World of Norms and Standards 


8.1.1 Setting Norms and Standards 


Norms and standards are linguistically distinguished only in Germany, while interna- 
tionally they are generally referred to as standards.°? The scope of norms and standards 
set by recognised standards organisations, such as DIN in this country, lies between laws 
or directives on the one hand, and particular business standards or company-owned 
standards (such as apple plugs) on the other, and can be illustrated by the following 
pyramid (Fig. 8 A.1): 


We es nn en a ns ee ee ee 
publicly 
state hostity accessible 
(À 
& directives 
& and decrees 
X- rn a u TE PAE GES: 
I aaa len x 
self-manage- 
ment of the not 
acon factory standards públi 


Fig. 8A.1: Pyramid of scope of standards and norms. PAS stands for publicly available specification. 
(Source: DIN / DKE) 


A DIN standard is a document that specifies requirements for organisations, products, 
services, or processes. For standards to be accepted, broad participation of so-called 
interested parties, transparency and consensus are basic principles. In theory, all those 


51 Iwould like to thank Michael Barth for his useful comments as well as Frank Schmiedchen, especially 
for his congenial editing and necessary but painful shortening of the chapter. More detailed explana- 
tions and additions, especially on the ongoing standardisation activities in the platform examples, are 
therefore reserved for a later publication. 

52 All ofthe following descriptions of DIN and DIN/DKE/VDE and international activities are taken 
from the public announcements of these organisations (especially freely accessible websites) and are 
therefore provided without more specific references. 
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interested in a topic are given the opportunity to participate and to contribute their ex- 
pertise to appropriate committees or to propose such committees. Before final adoption, 
draft standards are also made known to the public, which can formulate comments or 
even objections to be finally dealt with by the responsible committee. The experts in- 
volved in committees that draft standards should reach a consensus on the final content 
before they give their approval. Standards are reviewed at least every five years to meet 
the state of the art. 

A pre-standard is the result of standardisation work, which is not published as a 
standard by DIN because of certain reservations regarding the content, because of a 
different drafting procedure compared to a classic standard, or because of the Euro- 
pean framework conditions. Pre-standards offer the public the opportunity to use the 
results of standardisation projects that cannot be published as DIN standards, e.g. 
the revised DIN VDE V 0826-1 with detailed information on safety in smart home 
applications. 

The faster way to prepare a standardisation and publish initial regulations is a DIN 
specification (SPEC) according to the so-called PAS (Publicly Available Specification) 
procedure, the content of which is developed by at least three parties (without needing 
consensus) and then published by DIN. It provides manufacturers with sufficient secu- 
rity for market testing. 

DIN SPECs are listed, for example, on current topics (such as Building Information 
Modelling (BIM), blockchains, autonomous driving, artificial intelligence or digitalised 
parking). As the results of standardisation processes, DIN SPECs are strategic means of 
developing, establishing and disseminating innovative solutions more quickly. A SPEC 
should not conflict with existing standard(s). A SPEC can, however, be published as a 
supplement to an existing standard. A SPEC can be the basis for a new standard if it 
forms the basis for international standardisation projects proposed by national stan- 
dards bodies (NSBs) as national preparatory work.°? 

Official standardisation, on the other hand, aims to formulate, issue and apply reg- 
ulations, guidelines, or characteristics and should ideally be based on the established 
findings of science, technology, and experience (the so-called ‘state of the art’) and aim 
to promote benefits for society. In contrast to legally binding regulations, norms and 
standards are basically of a voluntary nature. Nevertheless, industrial practice shows 
that standards are very important and are applied for the sole purpose of securing 
market access for a consistent product. 

Standards are only indirectly legally-binding, if laws or legal ordinances (e.g. EU 
directives) adopt them or if contracting parties make the application of standards 
binding under private law in agreements. Even in cases where DIN standards have 
not been made part of a contract, or not prescribed by law, they can serve as an aid 
in dispute settlements (e.g. in questions of liability under warranty law, tort law or 
product liability law), in order to assess whether a manufacturer has complied with 


53 internationally then referred to as PAS 
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the generally recognised rules oftechnology and has applied reasonable care (see also 
Chapter 7 above). Compliance with standards thus offers a certain degree of legal pro- 
tection against possible liability. For this reason, voluntary regulatory procedures have 
proven their worth. Technical norms and standards are increasingly being developed 
and applied worldwide. Historically, norms and standards functioned as pure techni- 
cal agreements. In the last decades, an increasing number of standards have included 
socio-political aspects (such as on the topics of quality, environmental protection, and 
sustainability). 

Founded in 1917, DIN is the most important national standards organisation in 
Germany. In addition, the German Commission for Electrical, Electronic & Informa- 
tion Technologies (DKE), as an organ of DIN and the Association for Electrical, Elec- 
tronic & Information Technologies (VDE), is the organisation responsible in Germany 
for the development of standards, norms and safety regulations, especially in the field 
of digitalisation. 

Asa platform for electrotechnical standardisation projects, DKE is the central com- 
petence centre for electrotechnical standardisation and is therefore responsible for rep- 
resenting German interests in the European and international standardisation organisa- 
tions. These private institutions provide the administrative and logistical preconditions 
for standardisation activities, while technical content of norms and standards are de- 
veloped by external interest-driven experts, who traditionally come from industry and 
bring their respective company or association interests to reach balance of consensus. 

Depending on the topic, other stakeholders may also participate in such committees 
(e.g. government, trade unions, academia, standards users, consumers, and/or civil so- 
ciety). The national committees (usuallya maximum of 21 members) also represent the 
so-called mirror committees for international standardisation projects and nominate 
the national delegates to the international bodies from their ranks. In this respect, it is 
a kind of bottom-up approach to sub-governmental regulation. Since the early 1990s, 
socially relevant interests (environmental protection, climate change, and other sus- 
tainability issues, especially adressing the UN SDGs (Sustainable Development Goals)) 
have increasingly being considered in standardisation activities, in addition to technical 
and economic aspects. 


8.1.2 Institutional Interlocking at European and International Levels 


At the European level, there are three major European Standards Organisations (EOS): 
CEN (European Committee for Standardisation), CENELEC (European Committee 
for Electrotechnical Standardisation) and ETSI (European Telecommunications Stan- 
dards Institute). As a politically important European speciality, it should be empha- 
sised that CEN can also implement legally binding, so-called harmonised standards 
at the instigation of the EU Commission (see below). An overview shows the mirror 
structure of national responsibilities on the national, the European and international 
levels: 
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General Electrotechnology Telecommunication 


International 
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(regional) CENELEC 


German 


(national) DKE 
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Fig. 8A.2: Overview of national and international standards organisations with incorporated-framed 
responsibility of DKE / CENELEC / IEC, technical focal points relevant to digitalisation. (Source: DIN/DKE) 


e Founded in 1946, the International Organisation for Standardisation (ISO) is the 
central global association of standards organisations.°* 

e However, it is not responsible for the areas of digitalisation and electronics, which 
falls within the purview of the International Electrotechnical Commission (IEC). 

e The development of telecommunication standards falls within the purview of a 
UN organisation, the International Telecommunication Union (ITU). 


Together, these three organisations, ISO, IEC, and ITU form the WSC (World Standards 
Cooperation). 

In accordance with the Vienna and Frankfurt Agreements on Technical Cooperation 
between ISO and CEN, or IEC and CENELEC, duplication of work is to be avoided 
and simultaneous recognition as an International Standard and as a European Standard 
is to be achieved. Priority is given to ISO and IEC. In general, the principle of country 
representation applies to ISO participation. 


8.1.3 European Standards System: CE Marking and Harmonised Standards 


The EU is a globally unique special case with its own standardisation system, which is 
currently also highly relevant for the world’s first legal regulation of AI applications (see 
section 8.3). The former “New Approach” (NA) of the European Community (1985), 
which was updated in 2008 by the New Legislative Framework (NLF), pursues in the 
EU the basic approach of a state relief°° for the development of a uniform economic 


54 DIN has been a member of ISO since 1951, being founded in the former Federal Republic of Germany. 
55 Basic idea of the New Approach, i.e. clear separation between sovereign legislation and private stan- 
dardisation. 
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and goods traffic area as well as for the marketing of products on the basis of (EEC, EC 
and today EU) so-called directives and the associated CE marking. Not all technical 
details should and can be regulated top-down in detail for all member states: Only the 
framework conditions can be decided at the EU level, while the designs are the respon- 
sibility ofthe private EOS. 

Today, there are approximately 30 EU directives covering more or less large product 
groups (e.g. electrical equipment, medical devices, hazardous substances in electrical 
appliances, energy-related products). They are limited to so-called basic safety require- 
ments, while the technical specifications of the contents for the respective products are 
to be defined by technical rules (standards). The above-mentioned European standard- 
isation organisations CEN, CENELEC, and ETSI are responsible for the development 
of these standards. The CE marking has been defined with regard to the elements and 
the individual modules of the conformity assessment: All actors acting in supply and 
distribution chains must ensure that only products that comply with the applicable 
legislation enter the EU market (this is the placing on the market). The CE marking 
thus serves to control the permissible marketing of products and thus represents a kind 
of passport for products in the European internal market. However, it is not a seal of 
quality and is not aimed at end consumers.°® It is not maintained by DIN. 

Here, the otherwise unusual concept of presumption of conformity plays a central 
role. The manufacturer or the person placing the product on the market (e.g., in retail) 
marks the product clearly visible with a CE label. In doing so, it states that the product 
fully complies with all corresponding requirements. The presumption of correctness of 
this statement is now automatically assumed (keyword: presumption of correctness). If 
it subsequently turns out that this is not the case or that damage has occurred, this has 
immediate legal consequences. 

European standards (EN) are regulations that have been ratified by one of the three 
European committees for standardisation mentioned above. All ENs are the result of 
a public standardisation process. Here, too, the process begins with a standardisation 
proposal submitted by a national member of the European standardisation organisa- 
tions, such as DIN, or by the European Commission or by European or international 
organisations. The national standards organisations decide on adoption as a European 
standard in a final vote lasting two months. For adoption, at least 71 % of the weighted 
votes of the CEN/CENELEC members are required (unlike in ISO, where the rule 
is: one country, one vote). Ratification of a European standard takes place automat- 
ically one month after a positive voting result. Thereafter, a European standard must 
be adopted unchanged as a national standard by the national standards organisations. 
Conflicting national standards have to be withdrawn to avoid double standardisation. 


56 The CE marking is not a mark of conformity with standards, but an EU directive conformity mark 
with a function as a supervisory mark, which is intended, for example, to make it easier for the trade 
inspectors in the EU countries to check whether the products are being marketed (placed on the 
market) in a permissible manner. 
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In Germany, for example, each adopted European standard is published as aDIN EN 
standard with a national foreword. The national foreword serves as an additional source 
of information on the respective standard for the user of the standard and is prepared 
by the responsible German mirror committee.°” 


Harmonised and Mandated Standards 


The term (European) harmonised standard (eHN) has a definition established by the 
European Commission in the context ofthe New Approach (see above): 


e the standard has a mandate or standardisation request from the European Com- 
mission and EFTA to CEN, CENELEC or ETSI, and 

e the reference of the standard has been published by the European Commission 
in the EU Official Journal. 


All standards prepared by CEN, CENELEC and ETSI are the result of European har- 
monisation and in this sense are harmonised throughout Europe. However, only those 
which fulfil the two above-mentioned requirements are to be regarded as eHN within 
the framework of an EU directive and the legal definition of the European Commis- 
sion. Mandated standards result from a politically motivated and (co-)financed order 
(mandate) of the EU Commission to develop certain European standards. For more 
than 4600 standards, the EU and EFTA have issued so-called mandates or standardisa- 
tion directives to CEN, CENELEC and ETSI, largely under directives issued under the 
former NA and NLF, respectively.°® 


8.2 Norms and Standards for Digitalisation 


In Germany, the German government is promoting specialist discussions on the digital 
transformation, also with a view to setting corresponding standards. To this end, the Ger- 
man Federal Ministry for Economic Affairs (BMWi) has co-initiated ten platforms with 
respective focus groups, which are intended to develop the digital policy challenges as well 
as approaches to solutions from a technical and political perspective with a view to nec- 
essary standardisation activities for the digital transformation in business and society.°° 


57 https://de.wikipedia.org/wiki/Europäische_Norm 

58 https://www.eu-richtlinien-online.de/de/informationen/harmonisierte-und-mandatierte-normen 

59 Platform 1 “Digital Networks and Mobility,’ Platform 2 “Innovative Digitalisation of the Economy” 
Platform 3 “Industry 4.0,” Platform 4 “Learning Systems,’ Platform 5 “Digital World of Work,’ Plat- 
form 6 “Digital Administration and Public IT; Platform 7 “Digitalisation in Education and Science,” 
Platform 8 “Culture and Media,’ Platform 9 “Security, Protection and Trust for Society and the Econ- 
omy,’ Platform 10 “Consumer Policy in the Digital World.” The 14th annual ‘Digital Summit’ Nov./ 
Dec. 2020 was, due to Corona, the first purely virtual one - see also the extensive program of lectures 
and 3 forums in the brochure of the BMWi: Digital Gipfel 2020 - Digital nachhaltiger Leben (Berlin, 
Nov. 2020); the still physical 2019 Summit in Dortmund (end of Oct.) under the motto ‘PlattFORM 
die Zukunft’ in Zeiten der Plattformökonomie, had accordingly an annual focus topic ‘Platforms, 
introduced since 2016, see program brochure of the BMWi (Berlin as of Oct. 25, 2019). 
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Platforms 3 (Industry 4.0) and 4 (Learning Systems), which are of particular rele- 
vance to this book, can only be presented exemplarily in some detail.°° 


8.2.1 Industry 4.0 


The BMWi has committed to joint European projects, for example on the topics of 
microelectronics and communication technologies, which have various standardisa- 
tion interrelationships. The first edition of a so-called “Industrie 4.0” standardisation 
roadmap was published in 2013, the fourth and so far last Industrie 4.0 in March 2020 
(DIN/DKE, 2020).°' This explains the cooperation with the German “Plattform Indus- 
trie 4.0”, with regards to the fundamental innovation and transformation approach of 
industrial value creation and the 2030 mission statement for Industrie 4.0 (DIN/DKE, 
2020, p. 12 £.). The Industrie 4.0 platform was founded by three German industry associ- 
ations (BITKOM, VDMA, ZVEI) and is headed by two German ministries. In addition 
to standardisation, there are also the fields of action research and innovation, security of 
networked systems, legal framework conditions, and work and training. DIN is involved 
in contributing results at the international level. 

The following diagram (Fig. 8A.3) shows the interconnectedness of DIN’s activities 
of the key players shaping the digital transformation for Industrie 4.0: Plattform Indus- 
trie 4.0, Standardisation Council Industrie 4.0 (SCI 4.0), and Labs Network Industrie 
4.0 (LNI 4.0). 


BPLATTFORME 
„ılNDUSTRIE4.0 
* Recommendations for action and strategic approach 


* Small and Medium sized Companies (SMEs) 
* International Cooperation 


dünn STANDARDIZATION Biaid 
a COUNCIL transformation 
w H INDUSTRIE 4.0 EHLNI4. Oi. 
= Initiation of cross-sector PA er x Network of testlabs 

standards < | | | 5 = Practice tests 


= Coordination of national and x 
internal standards N 


= Strengthening international 
cooperation 


VA * Validated feedback of the results 
for standardization 


Fig. 8A.3: Interplay of central standardisation actors in Industrie 4.0. (Source: DIN®) 


The standardisation roadmap (NRM) addresses the implications of the fourth industrial 
revolution on the organisation and management of the entire value chain. The techno- 


60 On the socially controversial topic of platform 1 ‘mobility; for example, Sabautzki (2020) provided a 
political critique regarding ‘lobbying: 

61 Only exemplary outlines can be given here for this publication of around 140 pages. 

62 https://www.sci40.com/sci-4-0/iiber-uns/ 

63 https://www.din.de/de/forschung-und-innovation/themen/industrie4-0/arbeitskreise 
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logical merging of IT (Information Technology) and OT (Operational Technology) is 
leading to the overlapping of previously separate areas of standardisation.“ 

It is assumed that issues, requirements, and working methods that were previously 
relevant to the information and communication technologies sector are now increas- 
ingly affecting all sectors. The central questions here are: What will a global digital value 
creation system look like? How can the normative framework conditions for this be 
identified and implemented? The 2030 mission statement of the Industrie 4.0 platform 
proposes a holistic approach to the design of so-called digital ecosystems. The goal of 
Industrie 4.0 is to replace rigid and firmly defined value chains with flexible, highly 
dynamic, and globally networked value networks with new types of cooperation. Based 
on the specific prerequisites and traditional strengths of Germany as an industrial loca- 
tion, the framework of a future data economy is to be reconciled with the requirements 
of a social market economy. This model primarily addresses Germany as an industrial 
and business location, but explicitly emphasises openness and an orientation toward 
cooperation with partners in Europe and the rest of the world. 

Three strategic fields of action and their close interconnection are postulated as 
particularly central to the successful implementation of Industrie 4.0: sovereignty, in- 
teroperability, and sustainability. With the BMWi prestige project GAIA-X°®, the In- 
dustrie 4.0 platform has proposed a basis for a distributed, open data infrastructure for 
the whole of Europe based on European values. This project is being pursued in SCI 
4.0 (above diagram) with the aim of promoting interoperability. This is intended to 
ensure networking across company and industry boundaries, for which standards and 
integration are necessary. In addition, a uniform regulatory framework on decentralised 
systems and artificial intelligence is required so that companies and business models 
from Europe can be globally competitive. This is important for digital sovereignty and, 
above all, also offers trustworthy securities for users of cloud services. 

The former German Minister for Economy Peter Altmaier saw in this comparatively 
fast-moving initiative opportunities for an export hit for Europe, whose open ecosystem 
would also be joined by non-European users.°° 

Six working groups have been set up to implement the entire NRM Industry 4.0.° 

Here, for example, WG 1 Reference Architecture, Standardisation formulates its 
approach as follows: Standardisation in industry is not a new phenomenon, but In- 
dustry 4.0 is bringing about a major change. Standards that regulate a small section of 
production would no longer be sufficient for networked production. In the standards, 
hardware and software, user and supplier industries, and product design to recycling 


64 https://www.plattform-i40.de/P140/Navigation/DE/Industrie40/WasIndustrie40/was-ist-industrie- 
40.html 

65 (https://www.bmwi.de/Redaktion/DE/Dossier/gaia-x.html); (https://www.plattform-i40.de/P140/ 
Navigation/DE/Industrie40/ WasIndustrie40/was- ist-industrie-40.html) 

66 Thus again in the panel of the BDI Industry Day on 22 June 2021 ‘How can Gaia-X and European data 
spaces promote innovation?’ (live-stream: www.bdi.eu/tdi) 

67 https://www.din.de/de/forschung-und-innovation/themen/industrie4-0/arbeitskreise 
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mu 
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st be thought together. This is the only way that different components in digital 
systems can work together smoothly (keyword: interoperability). There are two focal 


points for this work: 
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a) onthe part ofthe “ISO/IEC Joint Working Group21” (ISO/IEC/JWG21) for the 
harmonisation of Industry 4.0 reference models with a Technical Report (TR); 

b) regarding the adoption of the standardisation proposal for the administration 
shell by IEC/TC65 to set the course for this to become the central integration 
plug for so designated digital ecosystems. 


on a) Industry 4.0 reference architecture model - RAMI 4.0 

The interaction and communication between factories with their machines tran- 
scends company and enterprise boundaries. For this reason, production companies 
from different sectors are to be comprehensively networked with suppliers, logistics 
companies, and othersin a value creation system. To achieve this, interfaces must be 
harmonised. This in turn requires internationally coordinated norms and standards 
for these interfaces. 

A reference architecture model is intended to provide a uniform conceptual and 
methodological structure as a basis for the experts involved from the various dis- 
ciplines to master complexity and speak a common language. It creates a common 
structure for the uniform description and specification of concrete system architec- 
tures. The Reference Architecture Model Industry 4.0 (RAMI 4.0) developed for 
this purpose in Germany represents such a model. This model has already been 
successfully introduced into the international standardisation landscape and pub- 
lished as IEC PAS 63088. 


on b) Management shell: structure and its submodules 

To ensure semantic interoperability®® of hardware and software components in 
production (machines, stations and individual assemblies within machines), the 
concept of the management shell was developed in Germany. In order to help the 
management shell to achieve a breakthrough in international standardisation, the 
concept was pre-coordinated with partners from France, Italy, and China, among 
others, under the coordination of SCI 4.0. With the acceptance of the standardisation 
application for IEC 63278-1 ED1 Asset administration shell for industrial applica- 
tions - Part 1: Administration shell structure at IEC/TC 65, a first step was taken to 
make the administration shell the central integration plug for the digital ecosystems 
described in this way. 

The GoGlobal Industrie 4.0 funding project also supports the global harmoni- 
sation of national Industrie 4.0 concepts through the SCI 4.0. International collab- 
orations that go beyond cooperation on the previous topics are intended to take 
account of the profound changes in organisational and value creation structures in 
the fourth industrial revolution (cf. Chapter 14). 


Definition of suitable data structures for the exchange of data and their defined meaning. 
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In view of the high level of interest in AI, an Expert Council for AI in Industrial 
Applications® was established in 2020 by the SCI to serve as a hub for normative dis- 
cussions and coordination in the designated area, both nationally and internationally. 


8.2.2 Platform Learning Systems — Artificial Intelligence — Standardisation 
Roadmap Al 


This 4th platform was initiated in 2017 by the Federal Ministry of Education and Re- 
search (BMBF) at the suggestion of the Autonomous Systems Expert Forum of the 
High-Tech Forum and the German National Academy for Science and Engineering 
(acatech). It is intended to serve as a venue for exchange and to promote the implemen- 
tation of the German government's AI strategy.” 

In seven thematic working groups, opportunities, challenges, and framework con- 
ditions for the development and responsible use of learning systems will be addressed. 
In addition to specific fields of application such as medicine and mobility, the focus 
is also on cross-cutting topics such as human-machine interaction or legal issues. At 
the beginning of 2018, the interdisciplinary Artificial Intelligence Working Committee 
was founded at DIN. In addition, DIN, together with DKE, prepared a white paper on 
“Ethics and Artificial Intelligence: What can technical standards and norms achieve?” 
on behalf of the BMWi. (DIN/DKE/VDE-without year). On Aug. 1, 2020, a steering 
group for a standardisation roadmap on AI was established under the leadership of the 
BMWi and DIN to pave the way for the expansion of Germany as an Al location. 

The setting of norms and standards also plays a central role in the German govern- 
ment’s artificial intelligence strategy” at all three levels (national, European, and inter- 
national), although this is primarily seen as a task for industry (BuReg, 2020, p. 41). In 
its update “Fortschreibung 2020” (Dec. 2020), the government announces its mandate 
to DIN/VDE (DKE) to have a comprehensive standardisation roadmap for AI drawn 
up, which should be presented together with company representatives, trade associa- 
tions and leading scientists at the Digital Summit 2020 (BuReg, 2020, p. 21). 

This NRM Al is to be a central building block in the AI strategy in order to open up 
international markets for German companies and their innovations. To this end, the NRM 
Alis to provide an overview of existing norms and standards on AI aspects and, in particu- 
lar, make recommendations with regard to future activities that are still necessary. It will be 
compiled by the respective interested parties from business, science, the public sector, and 
society. And DIN is to organise this cooperation in the sense of a neutral platform. In this 
respect, DIN is also to implement the German government’ Al strategy. Standardisation 
should promote the rapid transfer of technologies from research to application. 


69 See https://www.sci40.com/themenfelder/ki-k%C3%BCnstliche-intelligenz/ 

70 (https://www.plattform-i40.de/PI40/Redaktion/DE/Downloads/Publikation/Leitbild-2030-f%C3% 
BCr-Industry-4.0.html) 

71 www.ki-strategie-deutschland.de/2018 74 www.din.de/go/normungsroadmapki 
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The initial results of the BMWi-funded AI standardisation roadmap were presented 
at the Federal Government’s Digital Summit in November 2020 and handed over to the 
Federal Government as an overview of the status quo, requirements and challenges, as 
well as standardisation needs for seven key topics: Foundations, Ethics/Responsible AI; 
Quality, Conformity Assessment, and Certification; IT Security in AI Systems; Indus- 
trial Automation; Mobility and Logistics; and AI in Medicine.” 

To this end, five overarching and central recommendations for action were named 
(full recommendations and action requirements in the AI standardisation roadmap), 
the implementation of which is intended to build trust in AI and support the develop- 
ment of this future technology: 


e implement data reference models for AI system interoperability; 
e create horizontal AI basic safety standard; 

e design practice-oriented initial criticality testing of AI systems; 

e national implementation program “Trusted AT’; 

e analyse and evaluate use cases for standardisation needs. 


For this purpose, identified needs for topics (such as fundamentals or ethics / Respon- 
sible AI) were discussed in each case in workshops and participants as well as other 
interests were invited (at: Kuenstliche.Intelligenz@din.de) to participate in their im- 
plementation, i.e., in national investigations in further processes for which no times or 
processes had yet been announced at the time of writing. 


8.2.3 International Standardisation Approaches on Artificial Intelligence 


With a 35-member DIN mirror committee, Germany is thus represented on various ISO 
committees on Al. The responsible ISO subcommittee ISO / IEC JTC1 / SC42 (estab- 
lished in 2017) has already developed and published eight standards on AI. Another 
22 are under development. The secretariat is provided by the US-American National 
Standards Institute/ANSI and a US representative also serves as chair. It is worth men- 
tioning that at the international level (by ISO and IEC), the first Al-specific document 
published by these bodies in a joint JTC 1/SC42 was a Technical Report (TR) on AI 
and trustworthiness (ISO/IEC TR 24028 Information Technology - Artificial Intelli- 
gence- Overview of trustworthiness in artificial intelligence), with the aim of supporting 
standardisation activities on gap identification. 

It should also be emphasised that recently, for the first time in this field, a so-called 
management system standard (MSS) has been prepared for a future ISO 42001, whereby 
the end number “1” indicates the special feature that only these few MSS in the respec- 
tive series of standards with requirements for external certification have been prepared 
to date and are therefore regarded as flagships.”* 

72 www.din.de/go/normungsroadmapki 
73 The annual ISO surveys on generic MSS such as ISO 90001 on ‘Quality, ISO 14001 on ‘Environment; 


ISO 50001 on ‘Energy’ as well as on individual sector-specific MSS, e.g. in the automotive sector, show 
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World’s First Proposal by the EU Commission on the Artificial Intelligence Act (AIA) 


On April 21, 2021, the EU Commission presented a world-first regulatory proposal for AI 
in the form ofa Proposal for a Regulation of the European Parliament and of the Council 
laying down harmonised regulation on artificial intelligence (EU COM, 2021). With this, 
the EU COM intends to prescribe basic requirements for AI systems, for example on 
risk management, transparency, robustness, IT security, and human supervision of AI. 

The regulation would have immediate legal force and implies the corresponding 
mandates for harmonised European standards within the framework of the NLF men- 
tioned above (see section 8.1) by the European (private) standardisation organisations. 
In this respect, this Commission initiative is of extraordinary, strategic importance both 
for the EU and, where appropriate, for international follow-up activities. 

The AIA proposal expresses a risk-based approach that distinguishes four categories: 


risk-free AI systems should not be regulated; 

e low-risk applications (e.g., chatbots) are to meet transparency requirements; 

e dangerous AI applications (e.g., social scoring) are to be banned; 

e the intermediate, high-risk AI applications are to be marketed according to the 
principal of the NLF are to be placed on the market. 


For the technical concretisation of such requirements, reference is made to harmonised 
European standards (hEN), which are to be developed by the EOS (CEN/ENELEC 
and ETSI) on the basis of a corresponding standardisation mandate of the EU Com- 
mission. If these European standards are complied with by manufacturers, the fol- 
lowing will apply, as stated in section 8.1: it is presumed that this would also meet 
the requirements of the AIA, i.e. the so-called presumption of conformity exists. 
With the required, visible affixing of the CE marking, the conformity of the manu- 
facturer or the distributor with the applicable legal act and the corresponding eHN 
is declared and the product is placed on the European internal market with the CE 
marking. In this respect, therefore, the familiar procedure for the European system of 
harmonised standards with then considerable, worldwide first-time requirements for 
manufacturers of Al applications. DIN and DKE commented on this in a joint position 
paper on June 9, 2021 (DIN/DKE, 2021) under the heading: “Standards as a central 
building block of European AI regulation” The paper welcomes the fact that the Com- 
mission is following the NLF with this regulatory proposal, which has proven its worth 
for the European single market. As usual, all interested parties can participate in the 
standardisation processes to be set up for this purpose, which contributes to a high 
level of acceptance of standards on the market and at the same time prevents or helps 
to reduce non-tariff barriers to trade, since all national NSBs undertake to adopt these 


considerable global certification figures, especially for the generic ones (ISO 2019), which at the same 
time indicates the considerable business field activities that are triggered and associated with such MSS 
for consultants, certifiers, accreditation bodies. 
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standards unchanged in their respective national body of standards and thus also to 
withdraw any conflicting standards. 
In addition, however, demands are also made in the AIA proposal: 


e continuation and further development of the cooperation between the standard- 
isation bodies and the EU Commission; 

e the timely development of standardisation requests for AI by the EU COM in 
cooperation with the European Standardisation Organisations; 

e the deletion of Article 41 “common specifications’, which is included for a legal 
framework on AI and is criticised as non-specified enacting legislation”; 

e the involvement of the European standards organisations in the work of the ‘Eu- 
ropean Artificial Intelligence Board’ to be established. 


From DIN/DKE’ point of view, the German Standardisation Roadmap AI could serve 
as a basis for forthcoming standardisation work. The overview of existing norms and 
standards contained therein, as well as the listing of further standardisation require- 
ments, is described as a basis for forthcoming standardisation mandates. Practical work 
on the implementation of the AIA will have to show whether DIN/DKE will be success- 
ful with this. In any case, the AIA represents a major challenge for the interested expert 
groups to assume ethical and socio-political responsibility far beyond purely technical 
issues. This opens up an exciting field of observation and possibly also of activity for 
the Federation of German Scientists (VDW). 


8.3 Quo Vadis Digitalisation Standardisation? 


This chapter 8A has provided an inside view of the complex world of standardisation 
and has shown why standards and standard setting are also central to further digitalisa- 
tion. This was highlighted by the comments on Industry 4.0 and artificial intelligence. 
Initial experience shows trends for standardisation and digitalisation: 

At the national (DIN) level, the traditional rule-based standardisation bodies and or- 
ganisational processes can continue to be used effectively, also to successfully introduce 
German considerations into international (ISO) processes in preparation, which is also 
supported by officials. However, the traditionally private standardisation activities with their 
non-political self-organisation on the part of industry are being supported and promoted 
by the state to an unprecedented extent in the case of digitalisation topics, and the original 
bottom-up self-organisation of individual economic interests is thus being transformed into 
top-down influence by politically organised and controlled governance structures. 

The accompanying politicisation through the broad involvement of actors in the 
sense of multi-stakeholder processes is, on the one hand, democratically welcome, but 
on the other hand also poses problems in consensus-finding processes that should not 
be underestimated in the highly competitive environment, or raises suspicions of po- 


74 21-06_DIN_KE_position paper_Artificial Intelligence Act.pdf 
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litical legitimisation or showcase events or even lobbying activities, as criticised in the 
example ofthe Mobility Platform (Sabautzki 2020). 

There could be a tendency for a kind of culture to develop in ISO bodies, especially 
in the case of AI topics, to generate and already implement new work topics and pro- 
jects themselves from within the existing committees, instead of developing them from 
proposals from individual NSBs for international coordination processes, as envisaged. 
This can be countered for the corresponding ISO working structures by sufficient na- 
tional active participation and NWIP submissions/voting. Germany should implement 
the self-confidence it has expressed with regards to both Industry 4.0 and AI by consis- 
tently representing its interests, especially in the EU framework.” 

So far, European standardisation by CEN/CENELEC has not yet played an import- 
ant role for digitalisation standardisation (also due to avoidance of duplication of work 
according to the Vienna Agreement on ISO activities), but with the EU Commission, a 
political actor and standardisation mandator is present who can pursue the above-men- 
tioned legal powers, as now shown by its new AIA initiative, which takes up the so- 
cio-political mega-topic AI. Strengthening the European standardisation system is as 
imperative for the EU as an economic superpower with the largest single market in the 
world, as it is for achieving the EU’s goal of developing a digital and sustainable union. 

In general - without closer insights into other European countries and their national 
standardisation practices - it can be asked whether standardisation processes on such 
cross-cutting issues affecting the entire economy and society as digitalisation in Ger- 
many and at EU level could tend to outgrow the traditional private organisational pro- 
cesses and structures in favour of a hybrid character of state-political use and influence? 
Or can control by a private quasi-sub-state and voluntary self-regulatory instrument 
like the traditional private organisational structures of standardisation be preserved? 

In any case, the offensive efforts of international competitors, especially the USA and 
China, to assert their national standards on economic players in other countries require 
greater vigilance, precise analyses, and appropriate measures. The question is which struc- 
tures and processes help to assert the interests of the EU more effectively within Europe by 
strengthening the European standardisation system. But how can European interests also 
be strengthened through broad participation and assertion in international standardisa- 
tion processes? In this respect, it can be seen as a political exclamation mark that on June 
7, 2021, a public hearing was held by the Foreign Affairs Committee of the German Bun- 
destag on “Innovative Technologies and Standardisation in a Geopolitical Perspective’’°”” 


75 Wahlster, head of the steering group for the AI standardisation roadmap, confidently states: “AI research 
in Germany is among the global leaders. Norms and standards pave the way for developing innovative 
products from the results, which can become export hits for our economy” (Wahlster: ‘Künstliche 
Intelligenz: Ohne Normen und Standards geht es nicht, https://www.din.de/de/forschung-und-inno- 
vation/themen/kuenstliche-intelligenz). 

76 https://www.bundestag.de/auswaertiges#url=L2F 1c3 NjaH Vic3 NILZEwMy9BbmhvZXJ1bmdlbi 
84NDM2MjgtODQzNjl4&mod=mo0d538410. 

77 HBS 2020: Technical standardisation, China and the future international order - A European perspective 
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This forms the bridge to more geostrategic considerations with regard to norms and 
standards, especially outside the international rule-based standardisation processes, as 
will be presented in the following Part B of this Chapter 8. 


Postscriptum to Englisch Edition 


e Meanwhile, the DIN organised some meetings and communications on the 
Chinese ISO-strategies and practices, see: S. Gabler: China und die Normung. 
Ein Rückblick auf die DIN/DKE-China-Frühstückreihe, in: DIN-Mitteilungen 
1/2022 (pp. 11-18) 

e Further, referring to the new EU-Commission ‘Standardisation Strategy’ pre- 
sented 2 February 2022 (https://ec.europa.eu/commission/presscorner/detail/ 
en/ip_22_661), DIN published a first evaluation by S. Gabler: “EU-Standardisie- 
rungsstrategie veröffentlicht: viel Licht und wenig Schatten, in: DIN-Mitteilun- 
gen 3/2022 (pp. 7-9) 

e Concerning the German activities on Al standardisation, DIN/DKE is conduct- 
inga broad multi-stakeholder process in 2022 on the second Alroadmap in eight 
working groups (three of them new to the first one on: ‘Socio-technical systems, 
“Financial services’ and ‘Energy and Environment’) and additional subgroups. 
This roadmap will develop guidance for practical standardisations and will be 
presented to the next ‘Digital Summit’ in November 2022. 


B. Standardisation as a Geopolitical-Technological Instru- 
ment of Power 
Michael Barth 


The world of standardisation, which was originally remote from politics, has increas- 
ingly developed into an instrument of global influence, especially in connection with 
the increasing comprehensive networking through information and communication 
technologies. This is certainly due, on the one hand, to the already mentioned universal 
character of these technology areas, as they are cross-sectional technologies that affect 
not only individual sectors, but the entire economy, the state and society, and, on the 
other hand, certainly also to the underlying rules of information and communication 
technology concerning market and investment dynamics: 


e high innovation promises and strong growth rates entail disproportionately 
higher investments than in classic industrial sectors; 

e rapid growth leads to market dominance (“The winner takes it all”); 

e lock-in effect of the user as part of many business models. 
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In this context, the momentous differences between norms and standards in areas of 
information and communication technologies and standards in other technology areas 
must be carefully analysed. One major difference, for example, is that the effects of 
the extensive use of digital technologies that cannot be controlled nationally lead to 
dependencies much more quickly than would be the case, for example, with mechanical 
engineering components. The example of services available via cloud technologies il- 
lustrates this particularly clearly in comparison to mechanical engineering components 
that are important to industry: Once components have been installed in systems, they 
can easily run for several decades with the corresponding mechanical maintenance 
cycles. With added services using digital remote maintenance options, on the other 
hand, it is easy to exclude these components from predictive maintenance and control 
mechanisms for maintaining performance from one moment to the next. This becomes 
even clearer with purely digital services, which exist in B2C business just as much as in 
B2B. When payment ends, access to the service and thus also to the data used ends. This 
also occurs when systems have to be shut down due to an accident (or hacker attack). It 
is no coincidence that an important requirement of business customers for their service 
provider is data portability to other services, which global players in the digital industry 
often know how to prevent for lack of an alternative. 

In terms of security policy, this situation is made transparent by the example of sanc- 
tions or export embargoes: machines can continue to run for some time after sanctions 
have been triggered, and with a lack of spare parts, their performance decreases over a 
longer period of time. In the case of digital products, access ends from one moment to 
the next with the decision of the providing company or even the state behind it, which 
makes sanctions much easier to enforce and also more effective. 

The control of standardisation and thus the sustainable influence of standards in 
one’s own interest is thus suitable as a promising and effective instrument of power 
politics, especially in the field of digital and information technologies (cf. Rühlig, 2021). 

These connections manifest themselves in the struggle for digital sovereignty, which 
has now become a global phenomenon, especially in those regions of the world that 
are not home to global players in the digital industry. They are also evident where 
“cultural areas” of digitalisation meet, for example the United States of America and the 
European Union or the United States of America and the People’s Republic of China. 
Here it can be seen that these states are trying to establish their own digital “cultural 
spaces’, to demarcate them from one another and ultimately to expand them. This 
happens, for example, through regulation or social norms that attempt to control the 
use of digital technologies and services nationally, or to impose conditions that promote 
or force adaptation to the respective “digital cultural space” At the European level, for 
example, the General Data Protection Regulation (GDPR) should be mentioned here. 
In the People’s Republic of China, this is done through cybersecurity legislation, which 
promises the state and its administration full control over the technologies used. In 
the USA, for example, this is reflected in the procurement power of the public sector, 
which relies on “Buy American” not only for digital products and categorically excludes 
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suppliers from other legal systems. Variations of these tendencies can be found in all 
the systems discussed here. 

This clash of systems is evident not only in trade policy, technological, and cultural 
approaches, but also in the area of standardisation considered in more detail here. Eu- 
rope, as in many other subject areas, relies on a multi-stakeholder approach. As de- 
scribed in Section A by Eberhard Seifert, the European approach is strongly rule-based. 
Standards and norms are first discussed and drawn up at national level, then negotiated 
and adopted at European level, and only then taken to international bodies. 

Basically, the approach in the United States of America corresponds to the European 
approach. Here, too, there are large standardisation organisations that carry American 
developments to the international level. The main difference, however, is that large 
bodies such as the American National Standards Institute (ANSI) certify other bodies 
as standardisation bodies, thus creating a multitude of different standardisation organ- 
isations, currently more than 600 at ANSI alone. Each of these bodies can submit stan- 
dardisation proposals which, if they meet certain requirements, first become a national 
standard and are then brought to the international level. In this way, ANSI wants to 
create as much inclusivity as possible and also represent the diversity of the US economy. 
At the same time, the standardisation umbrella organisation ANSI also has the official 
task of representing US interests in important markets such as China or Europe, which 
means that an increasing mix between politics and standardisation can also be seen in 
the USA. This is also reflected in the historical function of standardisation organisations 
in the USA. For example,’® the National Institute of Standards and Technology was 
founded at the time to counteract the economic and technological superiority of Great 
Britain, Germany, and other competing systems.’? 

However, the practice of numerous smaller certified standardisation organisations 
also makes it easier for large, market-dominant companies to control the activities of 
individual standardisation organisations and to influence their results more strongly 
in their own interests. Thus, increasingly strong individual companies or consortia in 
the ICT economy determine the standards to which economic participants must orient 
themselves, especially if they have an interest in interoperability with the market leader. 
The United States thus relies here on the innovative power of the IT companies based 
there, which are pioneers in many areas. 

This means that digital technologies in many fields already elude the traditional and 
practised mechanisms and the committee culture that are usually found in the field of 
standardisation. Although the creation of norms and standards is still important here, 
the consensus within the community is much more strongly brought about by the re- 
spective market dominator or industry consortia. For example, it has a strong influence 
on the usability of certain certificate types in browsers when Alphabet stipulates which 


78 On the role and self-image of ANSI see: https://ansi.org/about/roles 
79 https://www.nist.gov/about-nist 
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Fig. 8.B.1: Comparison of the number of staffed secretariats in international standardisation bodies by 
China, USA, Germany and Japan in 2011 and 2020. 


certificates they accept for the Chrome browser. Meanwhile, this mechanism can also 
be seen in concrete national attempts to influence standardisation processes. 

One example that illustrates this particularly well is the People’s Republic of China. 
Formerly a pure production location for the world, it has outgrown this status both in 
its self-image and in its external perception. Its global power-political ambitions can be 
seen not only in its exchanges with other states, in the development and securing of raw 
materials worldwide, or in its dealings with influential technology providers (cf. Bartsch, 
2016). They are also reflected in the world of standardisation. For example, the number 
of secretariats and chair positions occupied alone shows a strong increase in Chinese 
ambition (cf. Rühlig, 2020). 

This is further underlined by the consistent application of Chinese representatives 
for vacant leadership positions in international standardisation bodies (cf. Steiger, 2020). 

This finds an even stronger expression in the strategic documents derived from the 
determinations of the medium and long-term plans of the state leadership. In direct 
relation to our topic, for example, the national standardisation authority has derived a 
Standardisation Strategy 2035. “China Standards 2035” is directly related to the national 
industrial strategy “Made in China 2025” and is intended to support and secure it in 
the medium term. All in all, these development plans are embedded in the proclaimed 
goal of the People’s Republic to be a world leader “politically, culturally, ethically, so- 
cially and ecologically” by 2050, the 100th year of its founding. Chinas head of state Xi 
Jinping formulated that the country must be the global leader in innovation by 2035 
(cf. Lamade, 2020). The foreign and economic policy of the People’s Republic is already 
causing concern not only among its direct neighbours, but is also forcing the European 
Union, for example, to rethink. For a long time now, China has not only been using 
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direct investments to secure influence in developing and emerging countries, but also 
member states such as Hungary, Greece, or Italy are exposing themselves to greater 
Chinese influence. 

While the standards for 5G, for example, were still developed in cross-industry 
consortia and providers such as Huawei used the existing standards in a large number 
of patents here, “China Standards 2035” is intended to establish the next step for the 
People’s Republic of China here, namely the setting of standards in important future 
topics such as information technology (here above all the fields of artificial intelligence 
and cyber security) (cf. Arcesati, 2019), biotechnology, high-end production, but also 
completely different important future fields such as environmental protection, agricul- 
ture, or the standardisation of urbanisation. 

With a total of 117 individual measures, China wants to achieve a top level of stan- 
dardisation both nationally and internationally. The main focus is on five fields: 


e strengthening the strategic positioning of standardisation; 

e intensification of standardisation reform and development; 

e strengthening the standardisation system and improving the capacity for leading 
high quality development; 

e taking the lead in international standards and improving the internationalisation 
of Chinese standards; 

e strengthening science management and improving the effectiveness of standard- 
isation efforts. 


All fields are aimed at strengthening the People’s Republic of Chinas ability to create 
standards and to make them valid not only nationally but also internationally. While 
many individual measures are simply about enabling the Chinese economy and the 
national standardisation system to create good standardisation and to use it to improve 
production in a wide variety of fields, the measures in the last two fields in particular 
are strictly geared towards converting national improvements into international stan- 
dardisation successes. In this context, great attention is also paid to state support for 
domestic research and international standardisation activities from the Chinese side. 
This is in line with the strategy of transforming China from the workbench of the world 
to a nation that creates so-called tier-one companies. According to Chinese diction, 
tier-one companies set standards, while second-tier companies develop technologies 
and third-tier companies merely build products for others. The Chinese leadership 
thus sees standard-setting as a way to control technologies and products. This in turn 
fits in as a technology-driven facet of the rule-based power projection of the current go- 
ing-abroad strategy of the People’s Republic of China (cf. de la Bruyere/Picarsic, 2020). 

The important role that the People’s Republic of China plays in high technologies 
is already evident in examples such as 5G, artificial intelligence®°, or the Internet of 


80 On standardisation in the context of AI and the US-China Balance Act: Ding, Jeffrey: Balancing Stan- 
dards: U.S. and Chinese Strategies for Developing Technical Standards in AI, Oxford 2020, available 
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Things. This is also evidenced by the suspicion that Western governments have of Chi- 
nas technological pioneering role - most recently observable in the ban on Huawei 
components in the UK’s 5G network or the debate about the IT Security Act 2.0 on 
the topic of critical components (cf. Schallbruch, 2020). The law does not mention 
the Middle Kingdom at any point, but it was clear to all participants in the discussion 
that took place beforehand that it was actually about how to keep “non-trustworthy” 
components out of German networks - for example, those of the market-dominant 
Chinese network suppliers. Thus, in the IT Security Act 2.0, it is not only the technical 
requirements in terms of trustworthiness that play a role, but also political assessments 
of the respective countries of origin of components. In the future, the Ministry of the 
Interior, with its IT security competence, as well as the Ministry of Economics and 
the Federal Foreign Office will be involved in the decision as to which components 
from private providers may be installed in economically important installations such 
as communication networks. This represents a paradigm shift in the economic and in- 
dustrial policy ofthe Federal Republic of Germany, which is otherwise oriented towards 
market-based mechanisms.” Recent involvement of the Bundestag’s Foreign Affairs 
Committee, which otherwise deals with rather non-technical topics, also fits in with 
this change (cf. Bundestag, 2021). 

The extent to which the Chinese intention will be crowned with success is not yet 
foreseeable. In general, it is in the interest of the international community of states, but 
also of the economy, to have Chinese companies at the “standardisation table”, as this 
promises a broader technological consensus and also a smoother procedure in global 
trade and technology transfer. The strong state interference, however, makes both the 
industry associations of the Western world and the decision-makers on both sides of the 
Atlantic uncomfortable, also because China has recently increasingly tried to enforce 
national standards in other nations that are beneficiaries of Chinese investments. In this 
respect, regardless of the prospects of success, another “battle of the systems” is to be 
expected here, in which norms and standards will be fought over.°? 

The newly enacted Chinese strategy on this topic has not yet had a direct impact on the 
reality of standardisation, because the “mills” of the standardisation bodies do not grind 
like the geopolitical situation. Nevertheless, with the described and self-documented am- 
bitions, the People’s Republic of China makes it clear in which direction it wants to go and 


online at: https://www.nbr.org/publication/balancing-standards-u-s-and-chinese-strategies-for-de- 
veloping-technical-standards-in-ai/ 

81 German industry also clearly expresses its concern about the increasing influence of the People’s 
Republic of China: Bundesverband der deutschen Industrie (ed.): Grundsatzpapier China, Partner 
und systemischer Wettbewerber - Wie gehen wir mit Chinas staatlich gelenkter Volkswirtschaft um, 
Berlin 2019, available online at: https://www.politico.eu/wp-content/uploads/2019/01/BDI-Grund- 
satzpapier_China.pdf 

82 On this conflict from the US perspective also extensive: Strategy of the United States towards the 
People’s Republic of China, published in German on the homepage of the US Embassy in Germany: 
https://de.usembassy.gov/de/strategie-der-vereinigten-staaten-gegenueber-der-volksrepublik-china/ 
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what the standardisation approaches ofthe United States and the European Union should 
be prepared for. Here, on the one hand, very rule-based approaches and, in part, highly 
dynamic standardisation intentions driven by individual companies stand in contrast to 
a strongly state-supported standardisation policy interwoven with geopolitical interests, 
which is equipped with extensive resources, both human and financial, and pursues a 
clearly formulated goal that is not necessarily that ofthe most interoperable technology. To 
this end, the US and EU power systems have to deal with many other global challenges (cf. 
Semerijan, 2016/2019). This will be all the more difficult in a system that relies on consol- 
idated assessment and agreement on norms and standards by many industry participants. 
Europe is thus also under pressure in this field due to its consensual approach between two 
system approaches (that of the United States and that of the People’s Republic of China). 
However, the strategic assessment of the European Commission with regard to standard- 
isation already clearly shows the need for action, especially in the field of information and 
communication technologies. Here the crucial fields are named and it is also underlined 
how important the influence on technologies is in a changing geopolitical landscape (cf. 
EU COM, 2021). In terms of speed of development and the build-up of technological 
pressure, the US and Chinese approaches are probably superior to the European system®?, 
which unfortunately also means for Europe: “Being caught between a rock and a hard 
place. However, the link between the political agenda and the standardisation agenda at 
European level alone shows that the need for action has been recognised. 


83 In addition, NIST is already carrying out initial activities to assess the Chinese activities and derive 
possible responses: https://www.nextgov.com/emerging-tech/2021/05/nist-wants-help-assessing-chi- 
nas-influence-emerging-technology-standards/174052/ 


Chapter 9 
Intellectual Property Rights 


Christoph Spennemann 


Introduction 


Intellectual property rights (or IP rights or IP for intellectual property) play an import- 
ant role in the creation and protection of technologies that make up the digital economy. 
Copyrights, patents, trade secrets, and designs protect, to varying degrees, computer 
software, digital platforms, information and communication technology (ICT), and 
their devices and applications. In addition, IP rights protect traded assets, e.g. digital 
music and literature. While digital technologies have developed rapidly, the legal frame- 
work for intellectual property lags somewhat behind. This means both opportunities 
and challenges for our societies. To ensure legal certainty, it is necessary to adequately 
adapt IP legal systems to the new technologies to ensure their use and development. 
At the same time, the protection of digital business models can marginalise traditional 
exploitation of ideas and endanger economic livelihoods. The adaptation of intellectual 
property rights to the digital world should therefore be done cautiously and allow nation 
states the necessary space to take into account their cultural and industrial specifici- 
ties. This is especially true for developing countries, which usually lack specific policy 
guidelines on how to strike a fair balance between digital rights holders and users. This 
chapter first provides an overview of the problems that arise in applying intellectual 
property rights to digital technologies. It then looks specifically at the role that IPRs 
play in the protection and use of economic data. 


9.1 Overview: Intellectual Property Rights and Digital Technologies 


IP systems were originally developed for the analogue age. Their basic purpose is to 
strike a proper balance between the interests of creators and inventors on the one hand 
and users and consumers on the other. This has become much more difficult in the dig- 
ital environment. On the one hand, it is technically possible to produce and distribute 
electronic copies of original works in unlimited quantities, which can threaten tradi- 
tional business in publishing, printing, and book trade. Furthermore, digital copies can 
be distributed across national borders, while intellectual property rights are restricted 
by national jurisdictions. On the other hand, there is the question of how the rights of 
consumers and competitors can be transferred from the analogue to the digital context. 
For example, someone who buys a patented or copyrighted physical product is free to 
resell it to third parties. Can someone who lawfully acquires a digital copy of a piece 
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of music or a film also sell it to others, within the technical possibilities of unlimited 
electronic reproduction?®? Also, to what extent is the reconstruction (reverse engineer- 
ing) of protected computer programs, which is essential for software developers, es- 
pecially in developing countries, comparable to simply reading a copyrighted book 
in an analogue context? Some developed countries have addressed these issues in the 
digital environment by limiting the scope that copyright offers in the analogue context 
(Samuelson/Scotchmer, 2001). 

The balance of interests between rights holders and users is not only important for 
digital technologies, but also for the data that can be generated from these technologies 
via online platforms. While data collected by a search engine cannot be protected by a 
patent, data can be protected as a trade secret under certain conditions, and copyright 
or certain sui generis database protection rights can be considered for the protection of 
data collections or databases (see the analysis later in this chapter). Policy makers face 
the task of striking a fair balance between incentives to create data and datasets (e.g. to 
promote Al applications) and the need to share data to promote big data analysis and the 
improvement of products and services. In addition, ways must be found to reconcile the 
potentially conflicting goals of innovation-enhancing data sharing on the one hand and 
data protection on the other as far as possible. Digital platforms can lead to difficulties 
in IP enforcement, but also in consumer protection. For example, are platforms liable 
for IP-infringing content by their users? The EU Directive on Copyright in the Digi- 
tal Single Market, adopted in 2019, met with considerable resistance in the legislative 
process because of its alleged obligation for platforms to install “upload filters” to select 
and block infringing content as distinct from non-infringing content. The discussion 
contrasts arguments for effective IP enforcement with concerns about automated deci- 
sions that replace human case-by-case consideration and may inadvertently ban content 
not protected by IP. 

Digital platforms and software producers have also raised concerns about abuse of 
dominance by some global companies. The competition proceedings launched by the 
EU Commission against Google and Microsoft are based on complaints about digital 
“lock-ins” that effectively tie consumers to a company’s products. Leading competition 
authorities differ on the conditions, under which holders of intellectual property rights 
in digital technologies are liable for abuse of a dominant position. 

In some cases where legislation has been deemed too slow, the private sector has 
tried to close the gap by introducing voluntary commitments in the digital context. 
Open source software (OSS) is based on copyright, but rights holders authorise third 
parties to modify and distribute the programme under certain conditions. This reflects 
the belief that in a sharing economy, the consumer can be a creator at the same time, 
contributing to the continuous improvement of the underlying technology. 


84 Rights holders have responded by increasingly licensing digital content rather than transferring own- 
ership, thus reserving the right to control the further distribution of the content (Okediji, 2018, p. 30). 
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A cooperative approach is also necessary when the development of new products 
depends on the interoperability of digital technologies from different rights holders. 
Interoperability is ensured by technical standards developed by standardisation organ- 
isations such as the International Telecommunication Union or private organisations. 
For example, mobile phone standards (most recently 5G) include a large number of IP 
rights. Standards developers rely on patent right owners to disclose all IP rights claims 
and provide licenses on fair, reasonable, and non-discriminatory (FRAND) terms. If IP 
owners hide their claims or refuse FRAND, the use of the standard is at risk unless IP 
laws or competition rules fix the problem. For example, the US Federal Trade Commis- 
sion sued Qualcomm, which owns essential standard patents related to 4G technology. 
The reason for the suit was Qualcomm’ refusal to grant its customers, such as Apple, 
access to 4G on FRAND terms. While this case illustrates the complex intersection 
between IP and competition law, the worldwide and multi-year conflict between Apple 
and Samsung over IP-protected technologies and external design of smartphones and 
tablets shows how courts in different jurisdictions can draw different conclusions about 
the infringement of software and device patents by competing technologies due to the 
territorial nature of IP law. 

In conclusion of this section, it should be mentioned that the questions addressed 
here are answered very differently - or not yet at all - in national legal systems. The mul- 
tilateral legal framework on intellectual property, the TRIPS Agreement of the World 
Trade Organisation WTO, provides the necessary scope for this. However, the TRIPS 
Agreement offers little guidance on how to shape national frameworks for digital value 
creation. This poses considerable challenges for developing countries in particular and 
justifies the need for multilateral exchange of experience in the WTO (cf. Brazil and 
Argentina, Joint Statement, 2018). 


Protection and Use of Data and Artificial Intelligence: What Role Do Intellec- 
tual Property Rights Play? What Is the Relationship to Private User Rights? 


This section will now examine the regulation of the innovation potential of data and the 
artificial intelligence (AI) generated by it. Data is treated as a broad term that includes 
“isolated or isolable entities that can be processed and analysed by machines”, such as 
statistics, financial data, measurement data, information available in lists, structured 
and unstructured texts, and multimedia productions (Swiss Federal Administration, 
2019-2023). 

Here, intellectual property rights as a classic instrument for promoting innovation 
move to the centre of considerations, since AI and data are not about tangible assets, 
but about the application of ideas through algorithms and their results. The decisive 
factor here will be how intellectual property rights are to be structured in order to 
guarantee legal certainty and investment protection on the one hand, but on the other 
hand to encourage data to be made available to other economic participants. The more 
willingly data is made available (in technical jargon, “shared”), the greater the innova- 
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tion potential ofthis data and the AI based on it. This approach is also the basis of the 
European data strategy published by the EU Commission in February 2020 (European 
Commission, 2020). In it, the Commission states the following (ibid., pp. 7/8): 


“The real value of data lies in its use and re-use. There is currently not enough data available for 
innovative reuse of data, including for the development of artificial intelligence. 


k] 

Despite its economic potential, data sharing between companies has not yet become sufficiently 
widespread. This is due to a lack of economic incentives (including the fear of losing competitive 
advantage), a lack of trust between economic operators regarding the actual contractual use of data, 
unequal bargaining power, fear of misuse of data by third parties, and a lack of legal certainty about 
who is allowed to do what with the data (e.g. in the case of jointly generated data such as that from 
the Internet of Things - IoT)? 


The following questions are analysed in this section: 


1. Are intellectual property rights suitable for protecting both AI and its underlying 
data? 

2. Do conflicts arise between the protection of economic operators through intel- 
lectual property rights on the one hand and the protection of users under the 
General Data Protection Regulation (GDPR) on the other? 


9.2 On the Applicability of Intellectual Property Rights 


9.2.1 Applicability of Intellectual Property Rights to Al 


e Patents protect certain inventions. Patent protection can be considered on the 
one hand for the Al itself; on the other hand for the products created by Al. Let 
us first consider the former . The crucial element of AI, the patentability of which 
is at issue here, is the algorithm. This is a procedure (digitally programmed in the 
context we are interested in) to solve a problem (Czernik, 2016). An algorithm 
thus falls under the category of programs for data processing systems, scientific 
theories, and mathematical methods, which are in principle not patentable.®° The 
European Patent Office therefore denies the patentability of AI per se. However, 
it affirms patentability if the AI in question and the underlying algorithm serve 
a concrete technical application (cf. Free, 2019/2020, p. 32).8° Algorithms alone, 
i.e. in abstract form, are thus not patentable. Similar to mathematical methods, 
the underlying idea here is that the general public should not be denied access to 
such building blocks of science and innovation through exclusive rights. This idea 
should also apply to the ideas and principles underlying a computer program, 
including the interfaces between different program elements. However, general 


85 For example, § 1 (3) of the German Patent Act. Comparable provisions can be found in the laws of 
other states. 
86 The US Supreme Court takes a similar approach (Levy/Fussell/Streff Bonner, 2019/2020, p. 31). 
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algorithms can be protected as trade secrets. Unlike a patent, this does not grant 
absolute copy protection, but merely protects against unlawful acquisition, use or 
disclosure of secret information by a competitor. However, the latter is not pre- 
vented from developing and using the algorithm itselfthrough fair business prac- 
tices (see further discussion below). Another peculiarity of patenting AI arises 
from the patent applicant’s obligation to disclose the invention “sufficiently clearly 
and completely to enable a person skilled in the art to carry it out.”®” Depending 
on the individual case, it is conceivable that AI may become self-executing to a 
certain extent through data-driven learning and that the patent applicant does 
not have full insight into the exact processes of the invention. In such a case, full 
disclosure of the invention is not possible. It is still open how patent offices deal 
with this problem. In the author's view, the duty to disclose the invention con- 
cerns the AI at the development stage, at which the inventor has developed it and 
can still fully understand it. It is this development stage, and not possibly further, 
independent ones, that must serve as the assessment basis for patentability, i.e. in 
particular the examination of novelty, inventive step, and industrial applicabil- 
ity. In this context, the International Association for the Protection of Intellectual 
Property has rightly expressed the view that this problem only concerns those 
inventions that consist essentially of Al itself, but not those that are merely made 
possible by AI, but then exist and function independently of the AI (such as the 
aerodynamic shape of a car body made possible by AI) (Association Internatio- 
nale pour la Protection de la Propriété Intellectuelle, 2020, paragraphs 7 and 8). 
This leads to the second question of this section, namely the patentability 
of products created by AI. In case of such inventions, the question arises as to 
whether they can still be attributed to the human programmer of the AI or its 
user, or are already to be regarded as the result of a self-determined AI. According 
to general opinion, German patent law presupposes a natural human being as 
inventor. Autonomously acting machines that independently produce something 
new through AI therefore do not qualify as inventors, so that a regulatory gap 
opens up for this area. The Federal Ministry of Economics still assumes that cases 
of such autonomous Al are extremely rare (BMWi, 2019, p. 24). But who should 
be entitled to the patent in the cases of non-independent AI that frequently occur 
today and through which an invention is created? The programmer of the un- 
derlying software? With respect to this and the underlying AI, the programmer 
may already have a patent claim. It seems excessive to grant him an additional 
patent on the Al-generated products. Companies that are to use AI to innovate 
would probably be reluctant to invest in such Al ifthey were not entitled to patent 
claims for Al-generated products (BMWi, ibid.). Accordingly, it is obvious to 
regard the user of the AI as the inventor of the product under patent law. Unlike 


87 Section 34 (4), German Patent Act. 
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88 


the programmer, the user has applied the AI to a specific individual case in order 
to create a specific innovative product or service. 

Copyrights expressly extend to computer programs to the extent that they are 
considered personal intellectual creations of the programmer.” By contrast, 
copyrights do not cover ideas and principles that underly an element of a com- 
puter program, including the ideas and principles that underly the interfaces. 
The requirement of personal intellectual creation must also be met by AI. Copy- 
right protection comes into consideration on the one hand for the Al itself - ie. 
an algorithm for the digital solution of a problem - and on the other hand for 
works produced by this AI - for example photos, music, texts, further software. 
While the protection of the AI itself would benefit the software developer, the 
rights holders to the works produced by AI would be users of AI, such as mu- 
sicians or developers of further software. The AI itself cannot be considered 
a personal intellectual creation (and thus protectable by copyright) in (rare) 
cases of independence, i.e. when its processes can no longer be understood by 
its programmer and are distinct from the basic AI. An exception only applies if 
the programmer has already provided for certain options of independence in the 
basic AI (cf. Schürmann/Rosenthal/Dreyer, 2019). According to current legal 
opinion, copyright protection requires a natural person as the rights holder - due 
to the requirement of personal intellectual creation (cf. Schönenberger, 2017). 
The legal situation is the same with regard to the protection of AI products. AI 
users who produce works such as music or texts with the help of AI can claim 
copyright protection for them. This no longer applies if the AI creates certain 
works independently and the AI user no longer has any influence on their design. 
(cf. Herfurth, 2019) As in the area of patent law, there is therefore a regulatory gap 
here. However, as already mentioned, this is irrelevant as long as the technical 
possibilities to enable independent AI remain limited. 

The right to protect trade secrets can extend to any AI. This is particularly in- 
teresting for AI that cannot claim copyright or patent protection for the reasons 
mentioned above. Of much more practical relevance than the hitherto rather 
rare cases of independent AI, these are in particular the ideas and principles of 
an algorithm, including the interfaces between different elements of a program. 
An EU directive from 2016 has harmonised the legal situation in the EU in this 
regard.” The programmer of the Al itself or a user of the Al can claim such 
protection. The right protects against unlawful acquisition, unlawful use, or dis- 


Section 2 (1) 1. and (2), Section 69a (3), German Copyright Act (UrhG), and Article 10.1, TRIPS 


Agreement. 
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Directive (EU) 2016/943 of the European Parliament and of the Council of 8 June 2016 on the protection 
of confidential know-how and business information (trade secrets) against unlawful acquisition, use and 
disclosure. (hereinafter: Trade Secrets Directive). 
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closure of the AI, in particular the underlying algorithm. However, the protection 
only exists as long as the AI is kept secret. If a valuable algorithm is directly 
part of a marketed product (for example, the software of a medical diagnostic 
device), a trade secret does not protect against attempts to fairly decipher the 
AI and in particular the underlying algorithm. This is a significant limitation 
of this legal institution. The situation is different in cases where the valuable 
algorithm does not become externally tangible, for example, it merely optimises 
an internal manufacturing process for a product. Here, a trade secret can offer 
valuable protection. 

e In summary, it can be stated that the existing system of intellectual property 
rights is in principle applicable to AI. The decisive factor in both patent and 
copyright law is whether the respective algorithm was developed specifically as 
a solution to a particular problem. In many cases, this will be the case. Abstract 
theories, principles, and ideas that are detached from concrete applications, as 
well as programme interfaces, are neither patentable nor copyrightable, nor 
are the rare cases of independent AI. Here, trade secret law only offers valuable 
protection if the corresponding algorithms or underlying ideas are not publicly 
accessible, e.g. they concern parts of an internal manufacturing process. 


Comment / Recommendations for Action 


In terms of legal policy, it seems desirable to promote the further development of AI in 
order to enable our society to reap the benefits of digitalisation, create promising jobs 
and secure Germany’s and Europe’s competitiveness. However, a future development of 
independent (“strong”) AI should be accompanied by ethical considerations as to which 
consequences of digitalisation are undesirable (cf. Schmiedchen et al. 2018). This would 
go beyond the purpose of this chapter. Intellectual property rights are crucial to deter- 
mining the focus ofinvestment incentives for AI. Exclusive rights play an important role 
here. On the other hand, especially in the ICT and software sector, exchange with other 
developers is also essential for innovation. Intellectual property law should therefore 
be appropriately balanced to promote innovation through both property rights and in- 
creased exchange. The exemption of the theories and ideas underlying algorithms from 
the scope of protection of the most important intellectual property rights enables their 
use by competitors and researchers. Traditional IP rights are widely applicable to AI, so 
important investment incentives are available in principle. However, there is a regula- 
tory gap for independent (“strong”) AI. Due to the current lack of technical possibilities 
to actually create and apply strong AI, there is no immediate need for action. However, 
consideration should already be given to whether and how strong AI should one day be 
protectable as an intellectual property right. Two questions should be addressed here, 
namely (1) How can intellectual property law be designed in such a way that the creation 
of strong AI itself proceeds along controllable lines? (2) Should intellectual property law 
be adapted to protect the products of strong AI? 
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(1) As already explained, there are significant concerns about the patentability or 
copyrightability of strong AI according to current legal opinion (among other 
things, due to the lack of a human inventor or author of an algorithm). Strong AI 
could, however, be protected as a trade secret. Would such protection encourage 
uncontrolled development of AI, which could lead to a scenario in which humans 
lose influence over AI? Unlike a patent, a trade secret cannot in principle prevent 
a competitor from independently developing the secret technology. A trade secret 
may be researched and deciphered as long as fair means are used and industrial 
espionage is not resorted to. This insight is important for the permissibility of 
reconstructing source codes that can provide information on the construction of 
computer software. This makes it possible to discover malfunctions and to con- 
trol the further development of an algorithm.” However, under current EU law, 
the reconstruction of software is not permitted without limitation, but is subject 
to certain limits determined by the copyright on the programme.* Accordingly, 
the decompilation of a programme is only permitted for the purpose of estab- 
lishing interoperability between the reconstructed programme and another inde- 
pendently created programme.” For the purpose of verifying the functionality of 
a programme, it may be observed, examined, and tested by loading, displaying, 
running, transmitting, or storing it.°* However, if these actions require even tem- 
porary or partial reproduction of the programme, the consent of the rights holder 
must be obtained.°° Since reconstruction of the source code usually requires 
copying of the programme (this can be inferred from: Samuelson/Scotchmer, 
2001, p. 1609), this consent requirement may prevent a programme review or, if 
a paid license must first be acquired, act as a financial deterrent. Not every author 
may be interested in having their software programmes reviewed. 


It has been pointed out in the literature that reconstructing the source code alone is not sufficient to 
fully understand how a programme works. Accordingly, further steps are required, which will not be 
discussed here (Samuelson/Scotchmer, 2001, p. 1613). 

Unlike the source code underlying the programme, which contains an idea that cannot be protected 
by copyright, a software programme represents the creative expression of this idea and can thus be 
protected by copyright. 

Article 6(2)(a) Directive 24/09/EC of the European Parliament and of the Council of 23 April 2009 
on the legal protection of computer programs. Available at https://eur-lex.europa.eu/legal-content/ 
DE/TXT/PDF/?uri=CELEX:32009L0024&from=en 

Ibid, Article 5(3). 

Ibid, Article 4(1)(a). The exception to the reservation of consent formulated in Article 5(3) is so broad 
that it does not create an actual exception but a circular argument: “The person authorised to use a 
copy of a program may, without having to obtain the authorisation of the rightholder, observe, study, 
or test the functioning of that program in order to determine the ideas and principles underlying a 
program element, if he does so by performing acts of loading, displaying, running, transmitting, or 
storing the program which he is authorised to do.” (emphasis by the author). It is self-evident that 
someone does not need the authorisation of the rights holder to do what they are entitled to do. What 
third parties acting without authorisation are entitled to do is clear from Article 4(1)(a), namely to load, 
display, run, transmit, or store the programme, but only so long as this does not require reproduction 
of it. 
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In view of future developments in strong AI, a revision of the EU Directive on 
the legal protection of computer programs seems necessary. What is needed is a 
clear exemption in favour of any activities that serve to verify the functionality 
ofa program. This seems necessary to critically accompany the future develop- 
ment of source codes and to prevent their functioning from being increasingly 
removed from human understanding and influence. 

Also from a legal policy perspective, the review of a programme should be 
able to take place independently of the author’s consent. Copyright protection 
does not extend to the idea underlying the creation,°® in this case the source 
code. Duplication ofthe program cannot be considered copyright infringement 
ifthe duplication is not the actual goal, but serves to understand the source code, 
which is not protected by copyright. A more restrictive application of copyright 
law also leads to an impermissible restriction ofthe freedom provided by atrade 
secret existing in a source code. Accordingly, the decryption of the protected 
secret by bona fide means is permitted. However, such decryption is made illegal 
if copyright law prohibits the reconstruction of the program for the purpose of 
its verification. 

In the section on the applicability of intellectual property rights to AI, it has 
been pointed out that the permissible decryption of a trade secret is only possible 
if the relevant technology is publicly accessible, for example, if it is integrated 
into a marketed product. However, if the AI is part of a purely internal manu- 
facturing process, for example, third parties have no access and the consider- 
ations just made on the revision of the EU Directive on the Legal Protection of 
Computer Programs offer little help. For such constellations, a proposal by the 
Federation of German Scientists (VDW) seems interesting. According to this 
proposal, Al-relevant design information and source codes should be stored in 
public institutions to make them widely accessible in the long term and thereby 
document malfunctions without gaps and optimise the chances of success for 
repairs (Schmiedchen et al., 2018, p. 17). Ifthe owner of a trade secret consents, 
no problems under intellectual property law are apparent. However, if there is 
no consent, the protection of trade secrets provides protection against public 
access for the purpose of maintaining competitiveness. There is one possibility 
of reconciling this property right with the documentation obligation proposed by 
the VDW in the area of research and development through public funds. State in- 
stitutions that provide public funding for the research and development of AI can 
impose certain conditions of use on the AI developer as a prerequisite for access 
to funding. One such condition could be the deposit of Al-relevant information, 
including certain algorithms. On the other hand, such a deposit requirement 
should be designed in such a way that it does not inhibit the willingness to invest 


96 See e.g. Article 9(2), TRIPS Agreement. 
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in the development of AI. This danger would exist if commercial competitors had 
unrestricted access to the deposited information. One could consider granting 
access against payment of a fee that compensates the AI developer for his efforts. 
Alternatively, access could be limited to non-competitors, such as researchers at 
public institutions. In addition, certain exceptions could be formulated, which 
would give a governmental control institution the right to publish deposited 
information, e.g. to avert a danger to certain public interests. The WTO TRIPS 
Agreement provides for a similar possibility in the protection of pharmaceutical 
test data, ifnecessary to protect public interests (cf. Spennemann/Schmiedchen, 
2007). However, there is no explicit exception in the TRIPS Agreement for AI 
and trade secrets. Here, a multilateral consensus seems necessary. It would have 
to be precisely defined to what extent there is a public interest in a filing obliga- 
tion for Al-relevant information and under what conditions this can restrict the 
recognised right to the protection of trade secrets. 


(2) The following options for protecting products generated autonomously by AI are 
already being discussed (BMWi, 2019, p. 29): 


e Leaving the status quo. In line with the idea expressed above about the im- 
portance of idea exchange and collaboration in the ICT/software sector, the 
independent development of innovation through strong AI could completely 
dispense with protection through exclusive rights. However, it should be 
borne in mind that other legal systems are already quite capable of protecting 
strong AI through intellectual property rights. For example, neither in the US 
nor in the British copyright system there is a requirement of a natural person 
as the author and a personal intellectual creation as the object of protection. 
Unlike in Germany, therefore, the products of strong AI can certainly be 
protected as the copyright of the person who created the conditions for the 
product to be produced (even if this is directly attributable to an independent 
AT).?” Less extensive protection could prove to be a competitive disadvantage 
for Germany. AI development cannot be reduced to the free exchange of 
ideas, but also needs investors. It therefore seems disproportionate to exclude 
any intellectual property protection in principle. Rather, it should be possible 
to decide on a case-by-case basis the degree to which one wants to rely on 
exclusive rights or open collaboration. An open source approach would offer 
this flexibility. However, such an approach presupposes the existence of an 
intellectual property right, which can then be disposed of openly or exclu- 
sively, as the case may be. 


97 E.g. Sec. 9 (3) UK Copyright Designs and Patent Act 1988; similar in approach 17 U.S. Code § 101, 
according to which the commissioner of a work (as opposed to the actual author of the work) can be 
the person entitled to protection. 
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e Creation ofa legal personality for AI. This option would allow for clear patent 
and copyright protection of autonomous AI and its works. The existence 
of legal persons illustrates that legal personality is not necessarily linked to 
human existence per se. On the other hand, legal persons are created by an 
association of natural persons. In this respect, one could be of the opinion 
that legal persons also carry certain aspects of human dignity because they 
represent a will expressed by a totality of natural persons (BMWi, 2019, p. 7). 
It seems doubtful that a comparable reflection of human dignity can also be 
assumed for AI. This question would first have to be investigated through a 
societal debate.°® In addition, important liability issues would have to be regu- 
lated. For example, an AI endowed with its own legal personality would have 
to have possibilities to answer for violations of rights caused by it through 
independent liability and payment of damages. Because of the complexity of 
these considerations, the creation of an Al legal personality does not seem 
desirable to the author at present. Less drastic but equally suitable measures 
exist to protect the products of strong AI. 

e Adaptation of the legal system. In line with the British or US model, the prop- 
erty right in products of strong AI could be awarded to the natural person who, 
as the last human link in the chain of command, has ensured that the AI can 
create such a product. This would mean abandoning the traditional require- 
ment of a personal-intellectual creation in the area of copyright. This could 
prove difficult to implement in terms of legal policy (BMWi, 2019, p. 26). 

e Creation of a new ancillary copyright. Less drastic than the adaptation of 
the German legal system (see above) appears the creation of an ancillary 
copyright for the results of strong AI. The institute of ancillary copyright is 
used in German law when a performance is to be rewarded that does not 
constitute a personal-intellectual creation. In particular, anyone who takes 
a simple photograph is granted such a right to the image, which expires 50 
years after the image is taken or appears.°° Since a photograph is not taken 
directly by a natural person, but by a photographic apparatus, copyright does 
not apply due to the lack of a personal-intellectual creation.'” The results of 
strong AI could be protected accordingly. 


Finally, it should be noted that the further development of AI depends not only on the 
protection of intellectual property rights, but also on the willingness of users to provide 
(share) data on which AI can be developed. In the European Data Strategy of February 


98 See in particular the discussions by Stefan Bauberger in Chapter 4 Machine Rights. 

99 See § 72 UrhG. 

100 In contrast to a simple photograph, a photograph is considered to be a photographic work protected 
by copyright if it fulfils heightened requirements of creativity and expression. In that case, it is not the 
function of the photographic apparatus but the creative use of the same by the photographer that is 
in the foreground of the assessment. Such photographs are therefore regarded as personal intellectual 
creations. Rights to such photographic works do not expire until 70 years after the death of the author. 
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2020, the EU Commission expresses concern about considerable deficits in the sharing 
ofindustrial data (see above). The applicability of intellectual property rights to data is 
problematic. This raises the question ofan appropriate legal framework for data sharing 
(see below). 


9.2.2 Applicability of Intellectual Property Rights to Data 


Aland data are closely linked. Without data, on the basis of which AI can be developed, 
Al is inconceivable. Legally, however, they must be kept apart. While the concepts of 
technical invention (patent law) and creative formulation (copyright law) seem quite 
obvious for AI algorithms, these connections are not obvious for data. Accordingly, 
there is currently no law in the EU that establishes a specific property right in data 
(cf. van Asbroeck/Debussche/Cösar, 2017, p. 22). Different variants of intellectual 
property rights come closest to such a protective purpose, even if particular problems 
arise from the fact that the system of intellectual property rights is not yet adequately 
adjusted to the new technical possibilities for the automatic compilation and analysis 
of data. 


e Copyright: Protection of individual data as well as data collections or da- 
tabases can be considered.’ This is not the case with data that is compiled 
solely on the basis of logical constraints and without any personal intellectual 
creativity, such as sports results or temperature measurements (cf. van As- 
broeck/Debussche/Cesar, 2017, p. 70). Similarly, data or databases that come 
about through automated processes, e.g. AI, are not eligible for protection. The 
lack of a property right can mean, on the one hand, that the owners of such 
data are unwilling to make it publicly available, as they cannot expect anything 
in return and lose control over it. In such cases, the innovation potential of the 
data remains untapped. On the other hand, if such data is already publicly avail- 
able, third parties do not need permission required by an intellectual property 
right to use it, e.g. in the context of scientific or commercial data analysis. The 
EU Directive on Copyright in the Digital Single Market, adopted in 2019, pro- 
vides for exceptions to copyright for the purpose of analysing legally accessible 
texts and data (text and data mining), as text and data mining plays a crucial 
role in scientific and industrial research as well as in the development of AI.’ 
However, rights holders can prevent the application of this exception to their 
works for the purpose of commercial research by means of an explicitly stated 


101 With further analysis of the national laws of some EU states. In Germany, it is proposed to derive a 
civil law property right to one’s own data from existing provisions of criminal law and civil law (ibid., 
p- 57). 

102 See e.g. § 4 of the German Copyright Act. 

103 Articles 3 and 4 of Directive of the European Parliament and of the Council of 17 April 2019 on copyright 
and related rights in the Digital Single Market and amending Directives 96/9/EC and 2001/29/EC. 
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reservation. This reservation has been criticised in the literature as putting AI 
developers, commercial research institutions, and journalists in the EU at a 
disadvantage compared to similar actors in the US (Hugenholtz, 2019). There, 
commercial text and data mining may be practised regardless of the will of the 
creators. The ideal would therefore be the creation of a data protection law for 
non-copyrighted data that encourages the publication of data and does not 
set too high barriers for use by third parties, even in the area of commercial 
research. 

Protection of databases: In addition to copyright, the sui generis property right 
for databases introduced in the EU in 1996 comes into consideration here." A 
“database” within the meaning of the Directive is a collection of works, data, and 
other independent elements.” The property right does not extend to individual 
data, but to databases in any form. Their makers can prevent the further use 
of the database or extraction of individual data by third parties. However, a 
prerequisite is that the producer has made a substantial investment in obtaining, 
verifying, or presenting the contents of the database. In this context, “content” 
is to be understood as an entirety that goes beyond individual data, or other 
elements. It has been pointed out in the literature that these requirements do 
not take into account the new technical developments in the field of AI and 
leave Al-created databases unprotected (cf. Van Asbroeck/Debussche/Ceésar, 
2017)". In the field of Al-automated data analysis, the content of a database 
is usually obtained by AI, i.e. automated. This does not require a substantial 
investment in the sense of the Directive, unlike perhaps the creation of the 
individual data itself. 

The right to protect trade secrets could give data holders a right to prevent the 
unlawful acquisition, use, and disclosure of the data under their control. The 
legal basis for this is the EU Directive on Trade Secrets already presented above. 
However, it also appears doubtful here whether data used in the context of AI 
analyses meet the protection requirements for trade secrets. To do so, they would 
have to have a commercial value, among other things.” This seems doubtful for 
individual data. Only in the context ofa data set do individual data become valu- 
able to show certain trends, behaviours, or developments. Data sets, in turn, are 
often shared among different actors in the context of an AI data analysis in order 
to abstract the data, compile it according to certain patterns, interpret it, and 
finally apply it as AI (van Asbroeck/Debussche/César, 2017, p. 123, Figure 5.1: 
Knowledge pyramid). If the different actors have not entered into a contractual 


104 Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protec- 
tion of databases. 

105 Article 1 of the Directive. 

106 This is different from Schürmann/Rosental/Dreyer, 2019, who seem to affirm a database protection 
right for the result of the data analysis, i.e. the AI. 

107 Article 2.1 b) of the Trade Secrets Directive. 
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agreement to keep the shared data secret from third parties, the shared data is 
no longer considered secret, which is a prerequisite for trade secret protection.'°® 
Contractual confidentiality agreements are therefore essential for the protection 
of data sets as trade secrets. 


Alternative Approaches to Promote Data Sharing 


As shown in the last sections, existing legal instruments offer limited protection for 
automatically generated data and may therefore be limited in their ability to encourage 
data sharing for innovation. Therefore, the creation of a law specifically tailored to 
machine-generated data has been suggested for some time. For example, the World 
Intellectual Property Organisation (WIPO) has initiated a consultation on the creation 
ofa separate right for data, which is still in its early stages in 2021 (cf. World Intellectual 
Property Organisation, 2019). 

An expert report prepared for the EU Commission proposes a non-exclusive right 
to data, which should be available to anyone who has demonstrably and legitimately 
processed or analysed certain data (see van Asbroeck/Debussche/César, 2017, p. 120). 
The non-exclusive nature of the right is intended to promote access to and exchange of 
the protected data. 

In its data strategy presented in February 2020, the EU Commission does not address 
this proposal further. The EU data strategy announces a legal framework for a common 
European data space for 2020 and 2021 (cf. European Commission, 2020, pp. 12-15). 
The focus is to be on data sharing and the creation of corresponding rights. However, in 
this context the Commission seems to attach more importance to the concept of open 
data than to a redefinition of existing EU laws on intellectual property rights. Specifically 
for the area of publicly produced data, the Swiss federal government also adopted a 
“Strategy for Open Administrative Data in Switzerland 2019-2023” in November 2018 
(cf. Swiss Federal Administration, 2019-2023). 


Potential Conflicts Between the Protection of Economic Operators Through 
Intellectual Property Rights on the One Hand and the Protection of Users 
Under the General Data Protection Regulation (GDPR) on the Other Hand 


Intellectual property rights of economic operators and rights of private users to their 
data under the EU General Data Protection Regulation (GDPR) differ in their objec- 
tive. GDPR user rights are not designed for economic gains of the right holders, but 
concern the control of the data provided. Essential elements of this control under the 
GDPR are the reservation of consent, the right to be forgotten, transparency and pur- 
pose limitation of data processing, as well as data portability. These are outgrowths of 
the right to informational self-determination, which in turn is derived from the general 
right to privacy. 


108 Article 2.1 a) of the Trade Secrets Directive. 
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Despite these differences, overlaps between the two areas of law are conceivable. 
Private user data is used by platforms for commercial purposes and, as outlined in 
the previous section, may be subject to certain intellectual property rights such as a 
trade secret, at least in connection with the data of other users as data sets (without 
any commercial involvement ofthe users being envisaged). In addition, AI developed 
on the basis of user data may be protected by patents, copyrights, or trade secrets (see 
above). 

Conflicts may arise here if the principle of transparency of data processing in 
Article 5(1) (a)GDPR'® is interpreted as entitling the user to disclosure of the data 
used for machine training or the algorithm used. If the latter are protected by a patent, 
reference can be made to the patent application, in which the technical details of the 
invention have to be disclosed in order to allow a replication by a skilled person after 
the expiry of the patent protection."° The ideas and principles underlying a computer 
program are excluded from the scope of copyright. Disclosure may therefore be required 
if it can be shown that an algorithm embodies a generally valid idea and not a creative 
expression of that idea. Such a demarcation appears difficult." However, this is not 
decisive in practice: in the 2nd chapter of this book, von Gernler and Kratzer emphasise 
that the algorithm is usually known in broad outline and that it is more important to 
disclose the data used to train the machine. However, these may be protected by trade 
secrets, where secrecy is at the core of the protection requirements. 

But also in cases where IPRs are not used to keep data secret, but on the con- 
trary to share data with other actors, conflicts with Article 5 GDPR may arise. 
Article 5 (1) (b) GDPRimposes the obligation to use personal data for a specific purpose, 
i.e. not to further process it in a way incompatible with the purpose for which it was 
originally collected." It is true that protecting data through intellectual property rights 
would create some legal certainty and thereby encourage data set owners to share them 
with other economic actors in line with the objectives of the European Data Strategy. 
However, this incentive to share data could be lost if at the same time there was a risk of 
a breach of Article 5 (1) (b) GDPR. Indeed, private sector representatives at UNCTAD 
e-commerce week 2020 stressed that the purpose limitation under Article 5 GDPR is 
a serious obstacle to further sharing of collected data (United Nations Conference on 
Trade and Development, 2020, p. 27, esp. 28/29). 


109 This provision stipulates that “personal data must be a) processed lawfully, fairly, and in a manner 
comprehensible to the data subject (‘lawfulness, fairness, transparency’ )”. 

110 Cf. Sec. 21 (1) 2nd, German Patent Act. 

111 The disclosure of the idea in the source code will usually require the decompilation of the protected 
programme. EU law sets certain barriers here. See above, Comment / Recommendations for action 
on the applicability of intellectual property rights to AI. 

112 This provision states: “Personal data must be [...] (b) collected for specified, explicit, and legitimate 
purposes and not further processed in a way incompatible with those purposes; further processing 
for archiving, scientific or historical research purposes in the public interest or for statistical purposes 
shall not be considered incompatible with the original purposes in accordance with Article 89 (1) 
(‘purpose limitation Y’. 
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Comment / Recommendations for Action 


The two conflicts between intellectual property rights and the GDPR described in the 
previous section will be addressed here. 


1. Can the holder of a trade secret be required to give it up in order to comply with 
the transparency requirement in Article 5 (1) (a) GDPR? 

2. Intellectual property rights in data are intended to promote legal certainty and 
thus the willingness to share data. Does the purpose limitation under Article 
5(1) (b) of the GDPR contradict this objective? 


In order to solve the conflict described under 1, the purpose of the transparency require- 
ment should be taken into account. According to Article 5(1)(a) GDPR, personal data 
must be processed in a way that is comprehensible to the data provider. How exactly 
an individual needs to understand the details of this processing depends, according to 
the author, on the purpose of the data processing. If the purpose is to create a profile of 
the individual’s commercial preferences in order to promote products or services more 
efficiently, it can be assumed that no essential legal interests of the respective individual 
are affected, especially since data is provided voluntarily in order to be able to access 
services that can be dispensed with if necessary, such as certain search engines. In such 
cases, a claim against a company for disclosure of its protected data records in favour of 
the transparency requirement seems disproportionate, as this would completely negate a 
recognised trade secret. Such a serious interference can only be justified if important legal 
interests are at risk and the individual does not have the choice to forego the data-based 
procedure. This is the case, for example, if data is used to assess creditworthiness, or 
in the case of sovereign examination of a claim by a government agency or a court (for 
example, in the context of expert opinions). In this context, the VDW has pointed out 
the need for transparency and traceability of Al-generated decisions (Schmiedchen et al. , 
2018, pp. 17-19). Neither EU nor German law currently explicitly states how a trade secret 
should be treated in the examples described above. The right holder will usually not be the 
state agency or a court, but a private entrepreneur who, on the basis of his protected data- 
sets, develops automated procedures that sovereign decision-makers use. Consideration 
should be given to imposing an obligation on the right holder to disclose records in such 
cases. However, for the sake of proportionality of the interference, competitors should not 
be able to benefit from this, but access should be limited to explicitly designated experts 
who check for misuse or biased use of the data for training purposes, without pursuing 
their own economic interests. In this way, the core of trade secrecy, namely securing 
an advantage over competitors, could be preserved. On the other hand, the disclosure 
obligation should not be made dependent on the existence of concrete indications of an 
abusive or biased use of the data. It would be difficult for the person concerned to prove 
this and would severely impair the effectiveness of the disclosure obligation. 

The conflict mentioned under 2. arises not only from the existence of intellectual 
property rights to data, but also from the sharing of data that is not covered by intel- 
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lectual property rights. Affected are both actors whose business model is based on vol- 
untary data sharing and those who need special incentives to share data, such as those 
granted by intellectual property rights, for the sake of legal certainty. Any incentive to 
share data, whether through intellectual property, sui generis rights, or open data models 
of the public administration," could run counter to the requirement of purpose limita- 
tion of data collection. Thus, there is a risk ofa conflict of objectives between promoting 
innovation on the one hand and data protection on the other. 

For shaping the digital future in Europe, it is crucial to limit such conflict and to find 
a practicable balance of interests. Both goals have their justification in promoting pros- 
perityand quality oflife and should not be seen as opposites but as mutually reinforcing 
factors. Innovation becomes more attractive ifit avoids disregarding individual rights. 
This can be achieved, for example, through anonymisation of data used or through the 
EU-wide recognised right to delete personalised data without calling into question the 
use of data sets in their entirety. Data protection that inhibits innovation in the long 
term indirectly benefits the strategic competitors USA and China. Rules established 
there, e.g. in setting new industry standards and exploiting private data, could set new 
global standards that contradict European values, but which Europeans would have 
to adapt to out of economic dependency if they themselves fell behind in the field of 
innovation. In more positive terms, a balance of interests between data protection and 
innovation could carry a specifically European model of society into the future. This 
balance of interests should strive to position Europe permanently alongside the USA 
and China as a global player and, through the attractiveness ofthe balance, to influence 
such countries that are in the second and third tier of digitalisation, such as Brazil, 
India, Indonesia, and South Africa. This requires a change of consciousness in the EU 
Commission as well as the European capitals, especially in Berlin, to understand and 
actively use the European social model not only as an EU-internal instrument for an 
“island of the blessed”, but as a global power factor. Only in this way will Europeans 
succeed in permanently shaping digitalisation to enable and promote a dignified life in 
the tradition of European individual rights. 


113 An example of this tension is currently provided by the use of patient data by health authorities in 
the context of the Covid-19 pandemic. The Swiss Federal Office of Public Health (FOPH) has been 
heavily criticised in the press for its refusal to publish case figures on new infections at the municipal- 
ity or postcode level, arguing that analysis of these data could help determine locations of increased 
new infections. The FOPH had invoked data protection because in small communities, published 
data could quickly be used to infer individuals. As a compromise, it has been suggested that only data 
from the postcode level should be published, which does not allow such conclusions to be drawn 
(Skinner, 2020). 
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Lethal Autonomous Weapons Systems — New Threats 
and a New Arms Race? 
Götz Neuneck 


Science and technology had an important influence on arms procurement, strategy 
development, and warfare, especially in the 20th century. The industrial and scientific 
revolutions of the 19th and 20th centuries have also politicised scientific fields, culmi- 
nating in World War II with developments such as long-range missiles, radar, operations 
research, or cryptography, some of which were decisive during the war. In the Cold War, 
the subsequent dangerous and enormous resource-devouring arms races between the 
two superpowers produced weapons of mass destruction that have not been completely 
dismantled or limited to this day (cf. Neuneck, 2009). The diffusion of these technolo- 
gies ensures imitation and replication by other states (proliferation). 

Three important scientific breakthroughs, in the late 20th and early 21st century 
have led to significant advances in the civilian sector, but have also found their way into 
military technology: 


(1) nuclear technology, 
(2) biotechnology, and 
(3) information and communication technologies (ICT). 


These technologies, including missile and space technology, are highly “ambivalent” and 
for a long time formed the core of the dual-use problem, which could be limited by in- 
ternational law, arms export controls, disarmament treaties, and ethical regulations, but 
not completely contained. Dual-use refers to the potential use of technologies, products, 
and scientific and technical knowledge for both peaceful and for warlike purposes. In 
the course of history, there have been many examples of scientific-technical knowledge 
being used in armament and war technology (cf. Forstner/Neuneck, 2018). 

In the 21st century, the focus of peace and security policy is on the field of infor- 
mation and communication technologies (ICTs), because here revolutionary develop- 
ments are taking place that are visible to everyone and are even reaching into the private 
sphere. Keywords here are the internet, smartphones, or autonomous cars. However, 
the ICTs developed in the broadest sense in the civilian sector are also leading to an 
increased digitalisation of armaments and war in the leading Western countries. 

Particularly, in the context of the debate on “hybrid warfare’, social dynamics trig- 
gered by information operations are being discussed that could become relevant to 
war. Today, this is understood to mean in particular a stronger networking but also 
autonomisation of weapon systems. The connection of sensors, various platforms on 
land, at sea, in the air, and in space, the rapid processing and forwarding of data enable 
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new operational profiles, missions, strategies and threat scenarios. However, this also 
can trigger new arms races, enormous arms expenditures and concerns about wars that 
violate international law or the lowering of thresholds for the use of force." 

A leading example for this trend is the development, procurement and deployment 
of unmanned systems, popularly known as “drones”. Today, drone missions are no lon- 
ger the privilege of the USA alone, but have become a serious threat to international 
security (cf. Horowitz/Schwartz, 2020). New revolutionary developments in autono- 
mous systems are to be expected, which will also enable new weapon developments, 
operational capabilities, missions, and war scenarios. This implies new challenges in 
the field of national and international security and peace policy, i.e. for procurement, 
training of soldiers, handling in the field, protection against hostile systems, prolifera- 
tion, and for arms control, arms exports, and ethical regulations for the prevention of 
war. In view of considerable investments in research fields such as pattern recognition, 
sensor technology, big data, robotics, cyber technologies, and artificial intelligence, a 
permanent arms technology impact assessment is necessary to understand the possible 
damage potential and the effects on peace and security more precisely. 

The debate on the purchase of combat drones for the German Army (Bundeswehr) 
illustrates the diversity of arguments regarding possible mission profiles of combat 
drones. While proponents merely argue that the drones would only protect their own 
armed forces, opponents speak, among other things, of a lowering of the threshold 
for war. However, weapons systems are not procured for narrowly defined scenarios, 
nor would a national ban automatically raise the war threshold or mean a lower risk 
of violence. In reality, the fundamental question is how future decisions in automated 
wars are to be made in accordance with international law and whether preventive rules 
can contain destabilising developments. 

In the first section, the current possibilities, uses, and problems of currently existing 
unmanned weapons systems are presented, while the second section is dedicated to the 
question of future developments in the field of autonomous lethal weapons systems. 
The final section addresses the question of how to limit or prevent possible destabilising 
developments (cf. Alwardt/Neuneck/Polle, 2017 and Grünwald/Kehl, 2020). 


10.1 What Are Drones and LAWS and What Developments Are Taking 
Place? 


Unmanned moving platforms have been around since the 1940s. The evolution from 
the first large ballistic missile, the V-2, to modern intercontinental ballistic missiles has 
been going on for a long time. Cruise missiles, on the other hand, use turbines, oper- 
ate in the atmosphere, and have become part of the modern military arsenal, as have 
torpedoes. These weapon carriers can only be used once and enable heavy payloads to 


114 See also: Frank Christian Sprengel: Drones in Hybrid Warfare: Lessons from Current Battlefields, 
Hybrid CoE Working Paper 10. 
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be transported with pinpoint accuracy over long ranges. The modern arsenal has been 
joined by unmanned aerial vehicles (UAVs), which land or take off automatically, can 
be steered remotely and can be used several times. Depending on their equipment, 
they can be deployed for long periods, enable flight times of hours to several days, and 
do not put pilots’ lives at risk. In the public debate, the term “drone” is usually used 
to describe anything that flies unmanned. Depending on the range and technology, 
different categories of drones can be distinguished. Drones with a fairly simple design 
have a low flight altitude and short range, while tactical drones are intended for medium 
flight altitudes and longer ranges. 

Initially, drones have been introduced in the military for reconnaissance and sur- 
veillance purposes, in many forms. The dual-use potential of this development becomes 
clear when one considers the various types of civilian drones used for recreational pur- 
poses or by the police, fire brigade, agriculture, science, etc. In each case, they can be 
controlled remotely by an operator on the ground, whether by means of a TV-link or 
pre-programmed. It stands to reason that remote control was first applied to aircraft, 
but unmanned land or water vehicles are also gaining in importance. Partial autonomy 
comes into play in some existing drone systems, as functions such as automatic take-off 
or landing can already be delegated and performed autonomously."> “° 


Armed Drones: Pandora’s Box Is Open 


Technically, it is obvious that these “passive” flying objects can also be actively armed. 
When they are armed, they are called “Unmanned Combat Aerial Vehicles” (UCAV)."” 
The step from reconnaissance to combat drone is obvious for the military, but non-state 
actors are also showing interest in these systems, as they supposedly enable “surgical 
strikes” at safe distances. Ifa country has a developed aviation industry, in-house develop- 
ments, licensed developments and imports from other drone manufacturers are possible. 

The first combat drones were used in the 1973 Yom Kippur War; since then, these 
technologies have been expanded, especially by the USA and Israel. The US Predator 
or Reaper combat drones were first used by the US in Afghanistan in 2001. The USA 
remains the trendsetter in this field to this day. The MALE drone programme (Reaper, 
Predator) was massively expanded and led to thousands of missions in Pakistan, Yemen, 
Afghanistan, Iraq and Syria and Somalia alone. According to an analysis by the Bureau 
of Investigative Journalism (BIJ), the USA carried out at least 14,040 drone missions in 
the four countries mentioned in recent years. The target here was often non-state actors 
in the context of anti-terror warfare. 


115 One also speaks of MALE systems: Medium-Altitude, Long-Endurance. 

116 The Bundeswehr has been using the “Heron” reconnaissance drone leased from Israel in Afghanistan 
since 2010 and in Mali since 2016. 

117 A distinction must be made between, among other things, multiple-use “combat drones’, which operate 
from a safe distance, and single-use kamikaze drones (“loitering munition”), in which drones fly over 
a combat zone, detect a specific target and destroy it. 
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A distinction must thus be made between attacks by the US Air Force in the context 
of combat operations or secret missions carried out by the CIA intelligence service 
(“targeted killings”). The latter strategy is highly controversial under international law 
and is still rejected by many states. Such missions require not only the aircraft itself, but 
also a prepared infrastructure, i.e. depending on the range: ground stations, encrypted 


data transmission, secure landing sites, a space component for data transmission, etc."® 


Tab. 10.1: State ownership, procurement, and use of advanced combat drones''? 


State of proliferation of combat drones 
y China, Georgia, Iran, Israel, North Korea, Pakistan, South Africa, Turkey, 
In-house production x 
= Ukraine, USA 
i) 
2 Egypt‘, Azerbaijan’, France“ United Kingdom’, Iraq‘, Italy*, Kazakhstan‘, 
2 Imported / leased Netherlands? * Nigeria‘, Saudi Arabia‘, Spain®', Turkmenistan‘, United Arab 
£ Emirates?“ 
Previous missions with the use of Armenia, Azerbaijan, Great Britain, Iran, Iraq, Israel, Nigeria, Pakistan, 
weapons Russia Turkey, USA, United Arab Emirates“ 
5 Germany”, France®, Greece‘, Great Britain, India, Pakistan, Russia, Swe- 
g | Development 4 . 4 ne . ? 
SE den‘, Switzerland’, Spain‘, South Korea, Taiwan, United Arab Emirates 
5 3 
6 g Import / Leasing Australia®®, Germany”, India? Jordans, Poland’, Switzerland? 


The following applies: ° U.S. makes, ° Israeli makes, “Chinese makes, ‘development as part of a consortium, ° de- 
velopment as consortium leader, ‘previously unarmed combat drone, ® uncertain information or not yet determined. 


In recent decades, more and more countries have invested in the purchase or develop- 
ment of their own combat drones (Tab. 10.1). A list from 2017 lists 35 countries with 
their own combat drones, with the USA, China, Israel, and Turkey being the main 
exporters. While only the US, UK and Israel had armed drones in 2011, the import of 
armed drones has increased dramatically since then. Between 2011 and 2019 alone, 18 
states have acquired armed drones, 11 of which use Chinese products.””° 

In 2019, 10 states had already used combat drones, often in a way that was illegal un- 
der international law. The US’ use of drones as part of the “Grand War on Terror” against 
non-state actors (Al-Qaeda and IS) was sensational, as was the assassination of Iranian 
Major General Qasem Soleimani in Iraq in January 2020. Other states have adopted 


118 The Bureau of Investigative Journalism’s Drone Warfare Project lists between 8,858-16,901 deaths in 
these four countries between 2010 and 2020, according to publicly available sources. 10 %-13 % were 
civilians. https://www.thebureauinvestigates.com/projects/drone-war 

119 Data sources include: Who Has What: Countries Developing Armed Drones. North America Founda- 
tion (NAF) World of Drones website. As of January 20, 2020. Available at: https://www.newamerica. 
org/in-depth/world-of-drones/4-who-has-what-countries-developing-armed-drones/ 

120 Under President Obama, the USA restricted the export of armed drones on the basis of the 1987 Missile 
Technology Control Regime and only supplied them to the UK and France under certain conditions. 
Under Trump, this regulation was relaxed and deliveries to Taiwan, the UAE and India were made 
possible. 
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these patterns, with Turkey carrying out drone attacks against the Kurdish Workers’ 
Party at home, Nigeria against Boko Haram and Iraq against the “Islamic State”. Saudi 
Arabia and the UAE are using combat drones in Libya and Yemen. Drone operations in 
the war in Syria and Libya by Turkey (e.g. in Idlib) and Russia have encouraged Azer- 
baijan to conduct similar operations for offensive purposes in the 44-day war against 
Armenia in 2020 in coordination with tanks and artillery. Sensitive US targets (UAV 
hangars, CIA buildings, etc.) have also been attacked in Iraq by “kamikaze drones” of 
Shia militias supported by Iran. Meanwhile, advanced attack drones are considered a 
serious threat by the US military’ (cf. Arraf/Schmitt, 2021). 

The Pandora’ box of the use of combat drones is thus open globally. Current de- 
velopments show that the USA is no longer alone in the combat drone market and 
has competition from China and Turkey. Today’s drones enable precision strikes even 
against individual tanks, make the battlefield more lethal, for example in Libya, Syria, 
and Armenia, and change the use of combat drones in favour of the attacker, as aircraft 
are freed up for other missions. 

In the German debate on the acquisition of combat drones, proponents argue that 
the Israeli drones to be acquired, the “Heron TP”, are to be purchased for the protection 
of the Bundeswehr. However, it has just been shown that combat drones can be used 
particularly well offensively against weaker opponents and also have a psychological ef- 
fect. They are particularly well suited as “intervention weapons” in asymmetric conflicts 
where air superiority is present (cf. Ehrhart, 2021). 

Although drones are referred to as “surgical weapons” due to their accuracy, which 
is technically correct at first, since the target can be fixed or hit more precisely than 
cluster munitions, civilian casualties are nevertheless by no means excluded. The cru- 
cial thing in alleged “surgical warfare” is always target planning in favour of avoiding 
civilian casualties, because collateral damage is always possible with any munition. An 
evaluation of publicly available sources shows that 10 to 13 per cent of US attacks also 
hit civilians. Local proximity of combatants in areas inhabited by civilians is also ac- 
cepted by combatants to cause civilian deaths, but initially does not change the technical 
definition of “collateral damage” of today’s weapons systems and the effort required for 
target planning under international humanitarian law. 

The combined use of cruise missiles and drones caused significant damage in the at- 
tacks on Saudi Arabia’s Abgaiq and Khkurais refineries in 2019, significantly disrupting 
oil supplies. The Houthi rebels in Yemen, backed by Iran, claimed the attack. However, 
the attacks were not from the south (Yemen) but from the north (Iraq) or east (Iran): 
25 unmanned aerial vehicles hit Aramco’s oil tanks and processing facilities in two 
waves in a well-coordinated and pinpoint manner on 14 September 2019. Apart from 
the surprise effect of this “coordinated multi-attack’, the stationed air defences (Skygard, 
Patriot) were also ineffective. In the future, such scenarios are very likely in war zones. 


121 Baku used Israeli “Harop” kamikaze drones to attack Armenia's radar stations and take out air defences, 
and Turkish “Bayraktar TB2” drones to take out air defences and armoured vehicles. 
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It should also be noted that it is, in principle, not so complicated to intercept today’s 
drones, once detected, as they fly slowly, manoeuvre little or cannot defend themselves. 
However, air defence is not prepared for this. It now stands to reason that considerable 
efforts will be made to develop future systems that are harder to detect, can evade or 
fly faster. The technological arms race will thus be further fuelled, as the next section 
will show. 

Besides the USA, European states are developing drones with stealth technology, but 
also other states such as China, Pakistan, India, Russia, Turkey are developing advanced 
armed MALE drones (see Table 10.1 and next section). 


10.2 What Future Developments Can Be Expected? 


As shown, the combat drones in use today are primarily remote-controlled, but some 
also have semi-autonomous functions such as automatic take-off or landing. However, 
the development of fully autonomous systems is at the top of the glossy brochures of 
drone developers in technically advanced countries. In the civilian sector, too, the de- 
velopment of self-driving cars or automatically landing aircraft, drones, and helicopters 
is being promoted with heavy investments. The superficial reasons given for greater or 
full autonomy are always to relieve the burden on humans and expected cost reductions. 
In the military sector, it is argued that autonomy is necessary in the event that no data 
connection to the aircraft is possible or in order to shorten reaction times in combat 
situations. In the battlefield of the future, more “autonomy” is expected for various 
functions, including target planning. 

This indeed raises the ethically relevant question of whether machines should and 
may decide on the killing of an enemy. Internationally, this has led to a debate, in which 
different parties and schools oppose each other. Related to this is the question of whether 
certain destabilising developments can already be identified, from which prohibitions 
must then be derived, for example through arms control regulations or bans under 
international law. First, however, it must be clarified how autonomous weapon systems 
(AWS) are to be more precisely characterised and technically realised. 

The further development of modern warfare is closely linked to the concept of Rev- 
olution in Military Affairs (RMA) (cf. Neuneck, 2011). RMA can be understood as 
the combination of weapons technologies, military doctrine, and the reorganisation of 
armed forces, so that the nature of warfare to date is fundamentally changing. Today's 
drivers for the advancing RMA discussion are: 


1. the structural changes in the international order, i.e. the preservation of the qual- 
itative military-technical superiority of the USA and the arms competition with 
China and Russia; 

2. the high investments in research and development (R&D) expenditures and 
military expenditures of the USA and and their scientific-technical, industrial 
support; 
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3. the dramatic development of information and communication technologies, and 
the 
4. integration of various technologies in force structures, training, and deployment. 


The Trump administration has coined the term “return of great power competition” in 
various key documents. This refers to the competition between the USA and China and 
Russia, into which the scientific and technological rivalry between these states is also 
increasingly being drawn. For example, the 2018 National Defence Strategy states: “The 
security environment is also influenced by rapid technological advances and the chang- 
ing nature of war: “advanced computing, “Big Data” analytics, artificial intelligence, 
autonomy, robotics, directed energy, hypersonics, and biotechnology” (Department of 
Defence, 2018, p. 3). 


A Central Question: What Are LAWS and What Control Are They Subject To? 


Lethal autonomous weapon systems (LAWS) are unmanned weapon systems or carrier 
systems that have weapons on board and are used for combat purposes. In certain op- 
erational phases, they are subject to only limited or no human control and are capable 
of operating in a targeted and partially autonomous manner in a complex dynamic 
environment. The autonomy of action is handed over to the machine itself. So far, the 
international community has not been able to agree on a uniform definition for AWS 
or a usable delineation of automated, semi-autonomous, and autonomous weapons 
systems. Table 10.2 is an attempt to characterise and distinguish the different terms, with 
the technological concepts building on each other. The distinction between partial and 
full autonomy poses particular difficulties.’ 


Tab. 10.2: Concepts for increasing autonomy that build on each other 


Term Use Examples 

Automated System follows pre-programmed com- Patriot, landmines, close protection against 
mands without variations ships 

Semi-autonomous Certain phases of an operation take place Brimestone (UK); Long-Range Anti-Ship 
fully autonomously Missile (LRSAM); various systems in 


planning and development 


Fully autonomous Adoption of human cognitive abilities in 
goal planning, tracking, etc. 


The lack ofa uniform definition represents an obstacle with regard to regulatory consid- 
erations and possible arms control policy approaches to AWS. However, since research 


122 The Pentagon had already presented a definition for an AWS in 2012 in a guideline: “A weapon system 
that once activated, can select and engage targets without further intervention by a human operator. 
This includes human-supervised autonomous weapon systems that are designed to allow human 
operators to override operation of the weapon system.” Department of Defence, Directive Number 
3000.09. Subject Autonomy in Weapon Systems, 2012. 
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and development are only just beginning here, it is difficult to estimate what future 
capabilities AWS will have at all (cf. Alwardt/Neuneck/Polle, 2017). Permanent arms 
technology impact assessments within the framework of preventive arms control with 
regard to R&D programmes are therefore necessary. 

AWS do not yet exist or cannot yet be clearly distinguished from increasingly au- 
tomated weapon systems with autonomous sub-functions. When examining future 
AWS, it makes sense to also include all those unmanned weapon systems that are in 
the planning or development stage that promise increasing automated or (partially) 
autonomous functions as well as capabilities and could be combined in parts or as a 
whole into an autonomous weapon system or an AWS system network in the future. 

In view of the expected developments, another problem becomes apparent here: 
the increased “autonomisation of war’, in which more and more sensors and weapon 
systems are connected to take over semi-autonomous functions on the battlefield. Also 
conceivable here is cooperation between one or more AWS and operators, i.e. the com- 
bination of manned and unmanned systems, in US jargon “manned-unmanned-team- 
ing” (MUM-T). This can bring a range of new capabilities and military advantages (such 
as longer endurance, higher speeds with much faster reaction times, and better environ- 
mental analysis for target selection). 

In the USA, there are various research institutions and universities in the field of 
robotics and AI that are engaged in R&D relevant to AWS (cf. Boulanin, 2016). Both 
the Pentagon and the armed forces have drawn up corresponding “roadmaps’, in which 
robotics and autonomy are described as key factors for further developments and pro- 
curement. In addition to research fields such as machine learning, big data, manufac- 
turing techniques, robotics, the focus is also on miniaturisation, swarm behaviour, and 
autonomisation.'”® According to the Pentagon, nearly $15 billion was spent on R&D, 
procurement, and operations of unmanned systems from 2016 to 2018. In 2019, the 
Pentagon established the Joint Artificial Intelligence Center (JAIC) to advance basic 
research, technology development, and military integration. 

The fear is that the strong research efforts also in the civilian sector (keywords: 
artificial intelligence and robotics), the future generation of “autonomous platforms’, 
e.g. by integrating automated and fully autonomous functions, could increasingly yield 
new capabilities and more effective weapon effects. This could also lead to new destabi- 
lizing scenarios. Technically, these changes could take place in particular at the level of 
communication, computing capacities, and software development, which is why they 
are particularly difficult to detect or control. 

In the USA, however, there were also civil society protests: thousands of Google em- 
ployees opposed the company’s participation in the DoD project “Maven”, which was to 


123 In October, the US military tested 103 Perdix micro UAVs (290g, 30 cm wingspan) that have been 
released from fighter aircraft and can communicate with each other. Because these networked systems 
can work together to decide whether the mission purpose has been fulfilled, such as reconnaissance 
of air defence systems, analysts assign an AI capability to the micro UAV. DoD Announces Successful 
Micro-Drone Demonstration, Release No. NR-008-16; 9 January 2017. 
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develop Al-supported methods for image analysis to identify, classify, and track persons. 
Concerns about the “lack of human control on the violent use” of LAWS have been also 
articulated in an open letter by 85 German computer scientists, which work in the field 
of Aland robotics.’* This demonstrates impressively that some civilian researchers are 
unwilling to work for military applications that are used for violent purposes. Instead, 
they advocate international regulations. 

The aforementioned R&D areas of Al, robotics, etc. are transformative technologies 
today, the main impetus for which is emerging in the civilian or commercial market, 
but which will also be taken up by the military sector. In some states, this will feed 
into the development of military weapons and doctrine, as well as transforming the 
respective aspects of future warfare. In an increasing number of countries, interest is 
already emerging in the future use of automated or autonomous weapon systems with 
advantageous military capabilities. These transformative technologies are seductive and 
problematic at the same time, which is why a precise armament impact assessment is 
necessary based on agreed principles. 

Development is not focused on drones or aircraft alone. Armed unmanned ground 
vehicles (UGVs) or armed unmanned surface vehicles (USVs) and underwater vehicles 
(UUVs) have so far only been developed by individual leading industrialised countries 
and have so far only been deployed on a very limited, militarily irrelevant scale. 

The US has long been the trendsetter in the development and deployment of LAWS. 
Key strategy documents emphasise the importance of autonomy, robotics, and artificial 
intelligence in new weapon systems for air, underwater/surface, or land operations. The 
so-called “Third Offset Strategy”, has budgeted $18 billion for R&D over the last five 
years.” Key areas here are autonomous learning systems, human-machine decision 
making. Specific UCAV programmes of the US Air Force are XQ-222 “Valkyrie” and 
UTAP-22 “Mako”. The US Army has presented a “Robotics and Autonomous Systems 
(RAS) Strategy”, in which various robotic systems (for reconnaissance, transport, and 
combat) are to be integrated into US Army formations between 2035 and 2040. In the 
UK, the defence sector is working on autonomy and stealth programmes for UCAV 
(Taranis), France is leading the nEURon technology consortium to build a European 
combat drone, also involving Italy, Sweden, Spain, Switzerland, and Greece. Initial test 
flights have been conducted. These demonstrators, as well as unmanned escort missiles, 
are part of the French-German-Spanish Future Combat Air System (FCAS) project, 
which will focus on both a sixth-generation manned multi-role combat aircraft and 
unmanned escort aircraft, as well as new weapons and communication systems. Other 
examples from Israel, Russia, and China could be added. 


124 Offener Brief: Initiative für ein internationales Abkommen zu Autonomie in Waffensystemen, spon- 
sored by the “Gesellschaft für Informatik, 1. November 2021, https://gi.de/meldung/gi-mitglieder-un- 
terstuetzen-initiative-fuer-ein-internationales-abkommen-zu-autonomen-waffensystemen-1 

125 The first offset strategy is the introduction of nuclear weapons, bombers etc. in the 1950s. In the 1970s, 
microprocessors, stealth etc. made new conventional weapons possible. See: Robert O. Work; Shawn 
Brimley: 20YY. Preparing for War in the Robotic Age, Center for a New American Security, January 2014. 
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What Are the Expected Uses of LAWS? 


Due to their capabilities and resulting advantages, LAWS will be particularly suitable 
for military operational scenarios that are difficult or too dangerous for human oper- 
ators due to environmental conditions. Examples include heavily defended terrain on 
land or so-called anti-access/anti-denial spaces that require very fast reaction times or 
a high degree of manoeuvrability for “air combat”. Characteristic here is a dynamic en- 
vironment without a permanent communication link. Covert operations behind enemy 
lines are also likely. With regard to propulsion, navigation, and communication, other 
technologies are required on land or in the air than, for example, underwater, in the 
mountains, or under rapidly changing weather conditions. 

The deployment of future LAW systems suggests changes in some areas of warfare, 
from which different security and peace policy problems and technological risks can 
be derived: 

Due to the dwindling human influence on concrete processes in combat operations, 
the associated increase in difficulty of assigning responsibility under international law, 
and the danger that the principles of international humanitarian law can no longer be 
adequately taken into account, the risk of the use of force increases in principle. Further- 
more, a lowering of the inhibition threshold for the use of force and the use of LAWS in 
the context of “anti-terrorism” actions or for the targeted killing of people could prevail.'?® 

Due to the high level of technology, LAWS are also more susceptible to external 
electronic interference such as jamming or spoofing or possible system manipulation 
(hacking). Unmanned systems already show a considerably increased risk of accidents 
and failure due to technical errors. Future machine learning combined with AWS and 
the independent addition or extension of their programming to an “extended artificial 
intelligence” also entail a risk of unpredictable behaviour and possibly “unpredictably” 
acting AWS. 

Misguided or unreliable LAWS may have destabilising or escalating consequences 
in a crisis. The accelerated pace of warfare associated with LAWS in automated war- 
fare, and thus a greater burden on operators in decision-making processes, can lead 
to a sharp reduction in necessary pilot deliberation time, which in turn can lead to an 
unintended escalation in a crisis. Moreover, LAWS in crises could also induce more 
proactive military behaviour and more dangerous operations. On the other hand, the 
risk of escalation to full-blown war may be lower in a combat operation with drones 
because information can be obtained more easily on the ground and rational decisions 
can be made without fear. 

The introduction of LAWS in a regional context may also have implications for 
regional or strategic stability: At the regional level, a military adversary would most 
likely react to a qualitative superiority in the field of LAWS by stronger armaments 


126 This argument applies not only to LAWS operations in war scenarios, but also in the context of “in- 
ternal security” by police and security forces. 
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or new military strategies. Regional armament induced by LAWS could thus have a 
destabilising effect on existing power constellations in the medium to long term and 
increase the risk of escalation in a region. Finally, LAWS could also enable new military 
deployment scenarios based on their potential capabilities (e.g. in the area of maritime 
warfare or in A2/D2 situations), which ultimately can lead to a shift in the balance of 
power in a region and thus, in turn, to new armament efforts. 

Of particular importance would be scenarios in which nuclear weapons are inte- 
grated or involved, be it that new attack options on nuclear targets are pursued by means 
of conventionally armed UCAV/LAWS, be it that new nuclear-equipped unmanned 
delivery systems are deployed to undermine an opponent’s second-strike capability 
within the framework of nuclear deterrence and thus endanger the strategic stability of 
deterrence. In this case, further nuclear disarmament would no longer stand a chance; 
nuclear modernisation or even rearmament would be the likely consequence. 

Due to the increase in efficiency and capability potentially associated with them, 
LAWS represent a future means of further developing state weapons potential. An in- 
crease in the military efficiency and striking power of the initiating state can therefore 
lead to competitors also increasingly procuring LAWS in order to either maintain the 
respective balance or to change this in favour of one side. The consequence would be in- 
duced offensive/defensive armament spirals and associated new operational doctrines. 
The danger of a regional or global arms race in the LAWS sector increases, combined 
with a considerable increase in military potential and the resulting risks. A technological 
race of drones and anti-drone measures is also to be expected. 

The majority of AWS hardware and software are, in principle, dual-use technologies 
that are largely freely available today and will be subject to few arms export restrictions. 
The development of very powerful LAWS will probably remain the preserve of a few key 
states for the time being. However, other states could develop less powerful LAWS or 
arm commercial autonomous systems or replicate existing or captured systems. In June 
2021, a UN report caused a stir, stating that the combat drone “Kargu-2” of the Turkish 
company STM had directly attacked soldiers with autonomous control in the Libyan 
civil war (cf. Cramer, 2021). It is not known whether any damage was done. The incident 
also illustrates that verification of the autonomy function is a significant problem if the 
drone is not physically available (cf. Kallenborn, 2021). 

This path is generally also open to non-state actors. The degree of autonomy of a 
system is largely determined by the programming, whose algorithms and programme 
components are for the most part not of decidedly military origin and are therefore 
difficult to control in proliferation. Furthermore, increased competition for the export 
of UCAVs can also be observed between some key states (e.g. USA, China, Russia), 
which poses the risk that existing export restrictions could increasingly erode and the 
proliferation of AWS could be accelerated. 


127 A2/AD means anti-access or anti-denial warfare. Defensive weapons or tactics such as land-mines or 
air defence are used to prevent an adversary from invading or traversing an area of land, sea or air. 
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10.3 How Can Future LAWS Developments Be Limited? 


The preventive containment, limitation, or complete prohibition of LAWS is in prin- 
ciple possible through ethical or (international) legal prohibitions, manufacturing, or 
proliferation provisions, arms export control regulations, or arms control treaties. In 
recent years, there has been a broad international debate and diplomatic as well as civil 
society initiatives on LAWS. This raises the question of whether it is possible in today’s 
international relations to preventively limit weapons systems that do not yet exist or 
to ban their use altogether, so that states do not even enter into their development and 
procurement or limit their use in the event of war. This is the task of preventive arms 
control (“ius contra bellum”) and international humanitarian law (“ius in bello”), each 
of which is based on different factual situations, principles, and instruments. It must 
therefore be clarified whether the international community and states can agree on new 
prohibitions, verification measures, and sanctions. 

So far, none ofthe existing arms control treaties directly prohibit, limit, or regulate 
LAWS in particular, simply because the corresponding technological developments 
were not even possible at the time they were drafted. However, both arms control law 
and international humanitarian law have developed a broad portfolio of principles, 
criteria, and measures (e.g. verification) that are in principle applicable to LAWS. It 
should be emphasised that it has been possible to ban certain types of weapons in five 
cases: Chemical Weapons (CW- Convention), Biological Weapons (BW Convention), 
Anti-Personnel Landmines (Ottawa Convention) and Cluster Munitions (Oslo Con- 
vention) as well as Blinding Laser Weapons (CCW). 


Arms Control and Arms Export Control: 


Disarmament and arms control primarily contribute to risk and threat reduction by be- 
ing oriented towards the balance of power of certain weapon systems and actors as well 
as their verification capabilities. Proven criteria are non-proliferation, crisis stability, and 
arms control stability, i.e. avoiding an arms race. They refer either to complete bans or 
the setting of upper limits of defined weapon systems or expected damage effects. One 
focus of the treaties established during the Cold War (see, for example, Table 10.3) was 
the limitation of weapons of mass destruction and their quite visible delivery systems 
such as missiles, bombers or tanks. The verification of these treaties is primarily focused 
on easily identifiable delivery systems. 

In the future, subjects of restraint or prohibition could also be autonomously acting 
weapon systems, which would thus in principle become part of the arms control process. 
The treaties listed in Table 10.3 provide for consultative commissions in which LAWS 
can be included in the treaty provisions. The 1991 Treaty on Conventional Armed 
Forces in Europe (CFE) covers the comprehensive limitation of five main conventional 
weapons systems in Europe, in which LAWS could also be integrated. Against the back- 
ground of the general arms control crisis, the CFE Treaty has been suspended since 
2007, but a new edition could draw on procedures and knowledge that have been devel- 
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Tab. 10.3: Arms control treaties and their applicability to LAWS (Alwardt/Neuneck/Polle, 2017) 


Arms Control Treaty Frame AWS part of the contract? Verification Status 

CFE Treaty multilateral | Probably AWS in general* Yes suspended since 2007 
Yes, autonomous UCAVs 
or UUVs; if they have in force (duration until 

New START bilateral characteristics of a strategic | Yes 2021; extension until 
system and are intended 2026) 


for nuclear armament. 


(Yes) has been 
Disputed whether UCAVs 


i 7 considered 
INF Treaty bilateral are to be equated with f Terminated 
i issiles, if applicable implemented 
cruise missiles, if app - | since 1991 
Yes, AWS in general, if they 
P are intended to participate 
CVEConvention and UN: in any way in the use of ie in force 
BW-Convention Framework y y i ; No (BWC) 
chemical or biological 
weapons 


* If AWS meet the 
definition of one of the 
major weapon types; 
the definitions there do 
not exclude unmanned 
systems (probably a 
matter of interpretation). 


oped and verified for years. This also refers to the broad set of instruments of the OSCE 
or the Vienna Documents, in which various specific, risk-reducing “confidence- and 
security-building measures” (CSBMs) have been negotiated and successfully applied. 

The prohibition of a certain degree of autonomy of weapons systems to be defined 
will hardly be realisable (or verifiable) on its own, since autonomous functions are 
essentially determined by electronics and software. It would be easier to regulate AWS 
either by specifying concrete conditions of use or by ensuring “meaningful human con- 
trol’, although it remains open what the characteristics of “meaningful human control” 
(MHC) are based on. The British NGO “Article 36” has elaborated and deepened the 
concept of MHC. International Humanitarian Law (IHL) (see next section) provides 
principles and criteria for this. For example, the practical application of Article 36 of 
Geneva Additional Protocol I in relation to the testing, development, procurement, or 
introduction of new LAWS could have a preventive effect. 

However, LAWS could also become part of preventive arms control and future (re- 
gional) arms control regimes (e.g. in Europe and South Asia or in the strategic context 
between e.g. the USA, Russia, and China) and thus contribute to crisis stability and in- 
ternational security. In the event of resuming or further development of the CFE Treaty, 
future military technology developments such as LAWS would also have to be taken into 
account. In addition, international efforts in the field ofnon-proliferation of AWS could 
be intensified, even if this is made considerably more difficult by the dual-use problem. 
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Many autonomy-relevant technologies arise in the civilian sector, cannot be banned, 
and therefore spread quickly. Only the integration into a LAW would characterise such 
a system as a weapon. 

Strengthening arms export regimes with regard to AWS systems or important (iden- 
tifiable) key technologies would be another important step here to prevent or curb the 
proliferation of dangerous military technologies. National arms export control is also 
guided by international regulations and contractually agreed restrictions on the sup- 
ply of strategically important military-relevant technologies. Examples of multilateral 
arms export agreements are the Missile Technology Control Regime (MTCR) of 19878, 
the Wassenaar Arrangement of 1994, the Arms Trade Treaty of 2013, or the UN Arms 
Register on Transparency and Confidence Building of 1991. LAWS could be included 
in national export control regimes. However, they are not part of international arms 
control law, as they are unilateral supply agreements of certain groups of states. 


International Humanitarian Law: the CCW Conventions and the UN GGE 


International humanitarian law (formerly: international law of war) primarily includes 
provisions of international law that aim to protect civilians, buildings, and infrastructure 
as well as the natural environment as best as possible from the effects of hostilities in 
the event of war or international armed conflict (“ius in bello)” (cf. Geiss, 2015). The 
four Geneva 